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INTRODUCTION 


Problem  Statement 

This  thesis  resulted  from  problems  identified  by 
the  Joint  Air  Force  Logistics  Command/Air  Force  Systems 
Command  (AFLC/AFSC)  Commanders'  Working  Group  on  Life  Cycle 
Costing  (LCC)  (35).  The  first  United  States  Air  Force  major 
system  acquisition  program  in  which  there  was  explicit  con- 
sideration given  to  LCC  was  the  A-X/A-10  Close  Air  Support 
Aircraft  competition  (27;  28).  The  A-X/A-10  competition 
addressed  LCC  considerations  via  an  operating  and  support 
(O&S)  cost  model  which  estimated  "operating  and  support  costs 
associated  with  each  proposed  design  [28:3]."  Proponents  of 
LCC  feel  that  LCC  techniques,  when  applied  early  enough  and 
over  an  adequate  period  of  time,  will  result  in  the  purchase 
of  more  reliable  and  cost  effective  systems  (38:i).  To  aid 
future  users  of  O&S  cost  models,  there  was  a need  to  assess 
the  effectiveness  of  the  initial  application  of  an  O&S  cost 
model  to  major  systems  acquisitions. 

Background  and  Justification 

Overview.  Within  the  past  decade,  the  obligation  of  funds 
by  Air  Force  planners  has  become  a subject  receiving  ever 
increasing  scrutiny  by  budget  analysts.  This  is  due,  in 
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part,  to  the  reduction  of  the  total  defense  budget  from 
46  percent  of  the  Federal  Budget  in  1960  to  26  percent  in 
1977.  Concurrent  with  the  budget  reduction  has  been  a sig- 
nificant increase  in  the  total  cost  of  acquiring  and  sup- 
porting new  weapons  systems  (23:16).  Analysis  of  total 
major  system  costs  has  revealed  that  only  40  percent  of  the 
total  dollars  expended  can  be  traced  to  research  and  acqui- 
sition procurements.  The  remaining  60  percent  is  absorbed 
in  O&S  costs  (28).  This  phenomenon  was  recently  summarized 
by  General  Bryce  Poe,  II,  in  a speech  given  to  the  American 
Defense  Preparedness  Association: 

O&S  costs  [have  begun]  to  significantly  reduce 
the  dollars  available  for  development  of  new  systems 
. . . the  result  [is]  clear.  Cut  operating  and  support 
costs  or  forget  about  development  and  acquisition  of 
the  new  weapons  required  to  meet  an  ever  more  serious 
threat  [23:16]. 


Life  cycle  costing.  A recent  advancement  in  the  Air  Force 
acquisition  process  has  been  the  application  of  LCC  tech- 
niques during  the  early  stages  of  the  process.  LCC  is  an 
attempt  to  determine  the  overall  costs  associated  with 
buying  and  owning  a major  system  so  that  the  most  reliable 
and  maintainable  system  can  be  acquired  (28) . The  applica- 
tion of  LCC  to  the  procurement  process  addresses  three  cost 
element  categories: 

1.  Acquisition  Costs — the  sum  of  the  unit  prices 
for  the  line  items  of  hardware,  data,  and  services 
being  procured. 

2.  Initial  Logistics  Costs — the  one-time  logistic 
costs  which  are  identifiable  and  would  be  incurred  by 
the  Government  for  the  item  being  procured. 


3.  Recurring  Costs — those  costs  incurred  by  the 
Government  in  connection  with  the  operation,  mainte- 
nance, and  management  of  the  item  being  procured  [8:1-4]. 

LCC  was  first  applied  to  the  procurement  process  with 

the  purchase  of  tires  for  the  F-100  aircraft  (27;  28). 


Operation  and  support  cost  model.  The  first  application  of 
LCC  to  a major  system  acquisition — the  A-X/A-10  Close  Air 
Support  Aircraft — utilized  an  O&S  cost  model.  The  value 
of  this  technique  is  its  ability  to  quantify  the  contribu- 
tion of  reliability  and  maintainability  (R&M)  parameters  in 
order  to  effect  a design  that  would  reduce  overall  logistics 
costs  without  adversely  affecting  the  desired  performance 
characteristics  (20:1;  35:2). 

The  model  consists  of  a set  of  mathematical  equations 
addressing  selected  cost  elements  which,  when  totaled,  yield 
an  approximation  of  the  total  O&S  costs  for  the  weapons 
system  over  a specified  period  of  time  (34:2).  These  ele- 
ments are  driven  by  maintenance  data  inputs  provided  via  the 
D056  Product  Performance  Collection  System,  a product  of 
AFM  66-1,  Volume  II,  Maintenance  Management , and  AFM  66-267, 
Maintenance  Data  Collection  System  (5). 

The  methods  utilized  to  compute  O&S  costs  are  pre- 
sented below.  R&M  factor  conversion  is  accomplished  via 
standard  formulae,  which  combine  the  many  facets  of  R&M 
costs  into  "logistics  effect"  dollar  figures  (8:2-4).  The 
general  formula  is: 
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where 


LE  = LEg  + LEr  + LEl  + LEg 

LE  = Total  logistic  effect, 

LEg  = Dollar  value  of  pipeline  spares, 

LER  = Dollar  value  of  base  and  depot  level  repairs, 

LEl  = Dollar  value  of  all  other  logistics  costs, 

LE  = Inputed  dollar  value  of  the  two  effectiveness 
attributes,  system  availability  and  system 
dependability  [16:2]. 

The  competing  contractors  apply  the  specific  formulae  to 
obtain  cost  predictions  called  Targeted  Logistics  Effects 
(TLE) . Source  selection  can  be  facilitated  through  compari- 
son of  competing  contractors'  TLEs.  After  system  production 
and  deployment,  the  government  uses  actual  data  on  logistics 
costs  in  the  same  set  of  formulae  to  develop  Measured 
Logistics  Effects  (MLE) . A comparison  can  then  be  made  to 
determine  the  amount  of  incentive  fee,  if  any,  to  be  awarded 
to  the  contractor.  This  is  accomplished  with  the  following 
formulae : 

Bonus  = (SR)  (TLE -MLE)  if  MLE  < TLE 

Penalty  = (SR)  (MLE -TLE)  if  TLE  < MLE  . . . 

. . . where  SR  is  a sharing  ratio,  i.e.,  percent- 
ages of  bonus/penalty  to  be  born  by  the  contractor  and 
the  government  [16:7]. 

There  are  two  factors  which  insure  that  the  con- 
tractor computes  TLEs  as  accurately  as  he  is  able.  First, 
the  TLEs  are  computed  during  the  competitive  phase  of  the 
acquisition  process.  Consequently,  the  contractors  incentive 


- 


4 


is  to  keep  the  TLEs  as  low  as  possible.  Second,  the  TLEs 
will  eventually  be  compared  to  the  MLEs  to  determine  the 
incentive  fee,  which  insures  the  contractor  will  not  make 


too  low  an  estimate  of  his  TLEs  (16:2-6). 


Application  to  A-X/A-10.  The  O&S  cost  model  used  in  the 
A-X  prime  contractor  competition  was  developed  in  Project 
ABLE — Acquisition  Based  upon  consideration  of  Logistics 
Effects  (20).  Table  1 identifies  the  areas  addressed  by 
each  MLE  in  the  Project  ABLE  model.  This  model,  dubbed 
"Strawman,"  was  incorporated  into  the  Statement  of  Work  (SOW) 
portion  of  the  Request  for  Proposal  (RFP)  (3)  which  was 
transmitted  to  the  competing  contractors,  Northrop  and 
Fairchild-Republic,  during  the  Competitive  Prototype  Phase 
(CPP)  of  the  A-X  competition  (8;  37) . The  United  States 
Air  Force's  planned  use  of  the  model  was  expressed  by  the 
Operations  Analysis  Office,  Headquarters,  Air  Force  Logistics 
Command  as  follows: 

. . . the  Air  Force's  intention  to  use  this  model, 
or  some  similar  model,  to  assess  the  supportability  of 
their  hardware,  and  further,  the  relative  supporta- 
bility will  be  one  of  the  criteria  on  which  the  pro- 
duction contract  will  be  based  (5:2). 

The  contractor  was  tasked  with  exercising  the  pre- 
diction mode  (TLE)  in  two  phases.  The  first  values  were 
to  represent  TLEs  after  reliability  and  maintainability  (R&M) 
of  the  Line  Replaceable  Units  (LRUs)  had  matured.  Secondly, 
value  estimates  for  5000  total  flying  hours  were  required. 

The  latter  was  expected  to  reflect  "early  learning  curve" 
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TABLE  1 


PROJECT  ABLE— MEASURED  LOGISTICS  EFFECTS 


MLE^  Dollar  cost  of  initially  procured  and  subse- 
quently replaced  LRUs  over  the  operational  life 
of  the  aircraft. 

MLE_  Dollar  cost  of  "off-equipment"  maintenance  on 
all  LRUs  removed  from  the  total  fleet  of  air- 
craft during  the  operational  life  of  the 
system. 

MLE.,  Dollar  cost  of  "on-equipment"  maintenance  on 
all  LRUs  serviced  during:  preflight,  post- 
flight, and  phase  inspections  and  time-change 
removals,  for  the  total  operational  life  of 
the  system. 

MLE4  Dollar  cost  of  "new  item"  inventory  incident 
to  spares  provisioning. 

MLE_  Acquisition  cost  of  Ground  Support  Equipment 
at  all  levels  of  maintenance. 

MLEg  Acquisition  cost  of  all  technical  data. 

MLE7  Acquisition  cost  of  training  equipment — 

excluding  maintenance. 

MLEg  Acquisition  cost  of  all  maintenance  training. 

MLE„  Dollar  cost  of  total  fuel  consumption  during 
the  operational  life  of  the  aircraft. 


(5:14-21) 
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characteristics,  while  the  former  would  produce  a consider- 
ably lower  cost  per  flying  hour  figure  (5:3).  The  contract- 
ing office  solicited,  "the  contractors'  best  efforts  ...  to 
identify  the  preferred  designs  of  the  A-X  configuration 
items  [5:14]." 

Under  this  methodology,  the  contractor's  TLE  is 
considered  with  acquisition  cost  in  making  the  pro- 
duction source  selection  decision,  and  the  TLE  of  the 
bidder  who  wins  that  contract  becomes  contractually 
binding  [5:4]. 

After  Fairchild-Republic  and  Northrop  Corporation 
received  the  "Strawman"  model,  the  government  requested  and 
received  contractor  recommendations  on  how  to  improve  the 
O&S  cost  model.  The  final  version  of  the  A-X/A-10  O&S  cost 
model  consists  of  thirteen  separate  cost  elements  (38:10). 
The  definitions  of  these  elements  are  listed  in  Table  2. 

Each  cost  element  was  determined  by  either  application  of  a 
formula  or  application  of  straightforward  accounting 
procedures  (8:D-10). 

Results.  The  results  obtained  from  the  application  of  the 
O&S  cost  model  to  the  A-10  program  were  to  serve  as  the 
basis  for  the  objective  evaluation  of  the  contractors' 
success  in  reducing  O&S  costs  (7:2).  There  were  three 
major  purposes  to  be  fulfilled  by  this  evaluation: 

1.  To  serve  as  a source  selection  criterion; 

2.  To  aid  in  evaluation  of  engineering  change 
proposals  (ECPs)  with  respect  to  their  impact  on  O&S  costs; 

3.  To  attain  a basis  for  the  determination  of  an 
award  fee  (7:2). 
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TABLE  2 

A-X  OSS  COST  TLEs/MLEs 


TLEj/MLE-j^ 

tle2/mle2 

tle3/mle3 

tle4/mle4 

tle5/mle5 

TLEg/MLEg 

tle?/mle7 

TLEg/MLEg 

tle9/mle9 

TLE10/MLE1q 

TLE-^/MLEj^ 

tle12/mle12 

tle13/mle13 


Initial  and  replenishment  spares  for  LRUs 

"Of f-equipment"  maintenance  of  all  LRUs 

"On-equipment"  maintenance  of  all  systems 

Ineffective  "off-equipment"  maintenance 

New  item  inventory  management 

Acquisition  of  Aerospace  Ground  Equipment 
(AGE) 1 

Acquisition  of  training  equipment  and  its 
ancillary  support  AGE 

Acquisition  cost  of  operational  phase  data 
Type  I training 
Fuel  consumption 

Spare  whole  engines  and  modules  (base 
supply) 

"Off-equipment"  maintenance  for  engines 
Spare  whole  engines  and  modules  (depot) 


‘AGE  is  currently  referred  to  as  Ground  Support 
Equipment  (GSE) . 


(7 :D-2) 
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The  values  used  for  TLE  computations  were  derived 


from  either  program  constants,  government  furnished  values, 
or  historical  comparisons  from  similar  weapons  systems 
(7:3).  The  values  of  TLEs  were  adjusted  after  the  valida- 
tion phase  to  account  for  differences  in  the  predicted 
growth  values  (learning  curve)  associated  with  maintenance 
(7:5).  MLE  values  were  computed  using  actual  data  compiled 
over  the  first  5000  hours  of  operational  flying.  The  first 
four  MLEs  were  computer  augmented  calculations  and  the 
remaining  nine  MLEs  are  manually  computed  (7:6-7). 


Objectives 

This  study  of  the  initial  application  of  an  O&S 
cost  model  to  a major  weapon  system  acquisition  will  pro- 
vide future  users  insights  into  model  utilization.  To 
accomplish  this  task,  the  following  objectives  were  estab- 
lished: 


1.  Determine  if  the  Air  Force  objectives  were  met 
in  the  initial  application  of  the  O&S  cost  model. 

2.  Investigate  methods  of  application  and  deter- 


mine if  improvements  are  warranted. 


Research  Questions 

To  fulfill  the  objectives  of  this  research,  the 
following  research  questions  were  postulated: 

1.  What  were  the  Air  Force’s  objectives  in  applying 
the  O&S  cost  model  to  the  A-X/A-10  acquisition  cycle? 
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2.  Was  the  data  required  by  MLEs  1-4  available 
from  the  designated  data  retrieval  system — D056? 

3.  Could  a valid  comparison  be  made  between  the 
TLEs  submitted  by  Northrop  and  Fairchild-Republic? 

4.  Could  a valid  comparison  be  made  between  the 
TLEs  submitted  by  Fairchild  and  the  MLEs  computed  by  the 
Air  Force? 

5.  Were  the  objectives  established  by  the  Air 
Force  accomplished  by  applying  the  O&S  cost  model? 

Scope 

This  research  effort  was  limited  to  investigating 
the  first  application  of  an  O&S  cost  model  to  weapon  system 
acquisition.  Specifically,  the  application  of  the  model  to 
the  A- 10  was  investigated.  Documentation  of  the  investiga- 
tion is  in  the  form  of  a case  study  of  the  A-10  application 
and  includes  a summary  of  findings  and  recommendations  for 
future  application  of  the  O&S  cost  model. 
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Chapter  2 

METHODOLOGY 

Introduction 

This  chapter  describes  the  methods  by  which  the 
research  questions  were  answered.  The  primary  assumption 
made  concerning  this  effort  was  that  the  O&S  cost  model 
itself  is  valid.  This  assumption  was  based  on  a validation 
study  conducted  by  the  United  States  Air  Force  Academy 
(USAFA)  (7) . The  areas  addressed  by  the  research  questions 
were:  Air  Force  objectives  in  using  the  model;  methods  used 

in  applying  the  model;  and  data  availability.  The  questions 
were  answered  based  upon  two  sources  of  information: 
interviews  and  documentation. 

Interviews  were  conducted  with  Air  Force  and  con- 
tractor personnel  who  were  directly  involved  with  applica- 
tion of  the  OSS  cost  model  to  the  A-X/A-10  program. 
Researchers  visited  the  A-10  SPO,  Air  Force  Acquisition 
Logistics  Division  (AFALD) , Air  Force  Business  Research 
Management  Center  (AFBRMC) , Sacramento  Air  Logistics  Center 
(SM/ALC) , Fairchi Id-Republic  Company,  and  Northrop  Corpora- 
tion to  conduct  these  interviews. 

The  document  research  included  a thorough  examination 
of  all  available  documents  pertaining  to  the  application  of 
the  OSS  cost  model  to  the  A-X/A-10  program,  including  SOWs, 
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Decision  Coordinating  Papers  (DCPs) , and  RFPs.  Additionally, 
documents  obtained  from  the  contractors  pertaining  to  the 
O&S  cost  model  were  reviewed. 

An  overview  of  the  areas  of  research  addressed  by 
this  methodology  is  illustrated  in  Figure  1.  As  shown  in 
the  figure,  comparative  analysis  of  the  results  of  applica- 
tion of  the  O&S  cost  model  may  be  favorable  or  unfavorable. 

If  favorable,  no  modifications  are  required.  If  unfavorable, 
several  areas  must  be  investigated.  This  research  addressed 
the  possibilities  that:  (1)  the  model  was  applied  incor- 
rectly; (2)  data  required  in  the  model  was  unavailable; 
and/or  (3)  the  objectives  set  for  model  usage  were  unrealis- 
tic. 

Air  Force  Objectives  in 
Applying  an  O&S  Cost 
Model 

Effectiveness  is  a measure  of  outputs  against 
accomplishment  of  goals  or  objectives  (26) . In  order  to 
assess  the  effectiveness  of  this  application  of  an  O&S  cost 
model,  it  was  necessary  to  determine  exactly  what  objectives 
were  established  by  the  Air  Force  to  be  met  by  applying  the 
model.  To  make  this  determination,  a representative  from 
the  research  team  discussed  these  objectives  with  A-10  SPO 
personnel,  and  personnel  from  AFALD.  Representatives  from 
Fairchild-Republic  Company  and  Northrop  Corporation  were 
asked  what  they  perceived  to  be  the  Air  Force  objectives  in 


Figure  1 

Areas  Addressed  in  Methodology 
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utilizing  an  O&S  cost  model.  These  interviews  were  supple- 
mented by  document  research  and  the  stated  objectives  were 
recorded  and  evaluated  in  terms  of  common  agreement  and 
clarity. 


Availability  of  Data 
from  D056 

The  data  retrieval  system  designated  for  use  with 
the  first  four  equations  of  the  A-X/A-10  O&S  cost  model  was 
the  D056  Product  Performance  Collection  System  (7:E).  This 
system  is  defined  in  AFM  66-1,  Volume  2,  Aircraft  Maintenance 
(Chief  of  Maintenance ) (36) . The  value  of  these  inputs  to 
the  O&S  cost  model  is  expressed  in  a previous  study: 

On  the  A-X/A-10  program,  the  two  contractors  con- 
sidered a combined  total  of  720  LRUs  in  their  pro- 
posals. Each  LRU  required  nine  input  parameters  in 
the  O&S  cost  model.  This  equates  to  roughly  6500 
elements  of  data  for  720  LRUs.  When  use  of  an  O&S 
model  is  proposed,  it  is  important  to  anticipate  the 
volume  and  availability  of  data  needed  to  implement 
the  model  [35:11]. 

When  these  factors  are  considered,  the  importance 
of  a data  retrieval  system  compatible  with  model  require- 
ments becomes  apparent. 

To  determine  whether  or  not  the  data  requirements 
for  the  model  were  available  from  D056,  the  research  team 
examined  the  data  elements  required  in  the  model  and  com- 
pared them  to  the  data  elements  available  from  D056 . The 
basis  for  answering  research  question  2 on  the  data  availa- 
bility affirmatively  was  whether  or  not  data  could  be 
obtained  from  this  system  and  applied  directly  to  the  model. 


Faulty  Application 

Validity  of  comparisons  of  contractors'  TLEs . To  test  the 
validity  of  comparisons  between  contractor  TLEs,  the 
research  team  examined  the  techniques  used  by  Fairchi Id- 
Republic  and  Northrop  in  exercising  the  O&S  model.  Specifi- 
cally, the  data  base  and  methods  used  in  LRU  selection  must 
have  been  consistent  between  the  contractors.  Consistency 
in  these  areas  is  essential  for  valid  comparisons  between 
the  TLE  computations  of  Fairchild  and  Northrop.  This  con- 
cept is  illustrated  in  Figure  2.  The  following  criteria 
were  used  to  determine  whether  or  not  consistency  existed 
in  the  areas  of  data  and  application  techniques: 

1.  The  Air  Force  must  have  used  the  same  data  base 
in  providing  data  to  Northrop  and  to  Fairchild  for  TLE  compu- 
tations. 

2.  The  same  criteria  must  have  been  used  by  both 
contractors  in  selecting  LRUs. 

If  the  contractor  did  not  draw  upon  the  same  data 
base  for  calculating  TLEs,  any  attempt  at  comparing  these 
TLE  values  would  not  be  valid.  This  was  the  basis  for 
criterion  number  1 above. 

The  use  of  the  same  criteria  for  LRU  selection  is 
essential  to  insure  that  TLE  comparisons  between  contractors 
encompass  the  same  baseline.  For  example,  the  contractors 
may  designate  LRUs  differently  so  long  as  the  LRUs  selected 
account  for  the  total  OSS  costs  for  the  entire  aircraft  (7:7-8). 
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THEN: 


TLE  COMPARISON  -*•  INVALID 

Where : 

F = Fairchild 
N = Northrop 
= Consistent 
DATA  = Data  Base 

LPU  DETERMINATION  = Criteria  used  to  select  LRUs 


' Figure  2 

Comparison  of  TLEs  Submitted  by  Contractors 

. 
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If  tho  entire  aircraft  is  not  accounted  for,  the  contractors 
may  decide  to  use  the  criterion  of  encompassing  high  cost 
drivers  using  the  same  dollar  cutoff.  Another  option  might 
be  for  the  contractors  to  select  only  those  LRUs  related  to 
a specific  subsystem  of  the  aircraft,  such  as  avionics. 

These  examples  serve  to  illustrate  that  innumerable  criteria 
are  available  for  LRU  selection.  It  should  also  be  apparent 
that  significant  problems  would  be  encountered  if  Fairchild 
chose  to  evaluate  only  those  LRUs  associated  with  the 
avionics  package  while  Northrop  decided  to  evaluate  high 
cost  drivers.  This  explanation  provided  the  basis  for  the 
establishment  of  criterion  number  2.  Violation  of  either 
criterion  will  result  in  the  inability  to  make  a valid  TLE 
to  TLE  comparison.  The  elements  of  this  investigation  were 
provided  via  interviews  with  the  contractors  and  by  docu- 
ment research. 

Validity  of  comparisons  between  TLEs  and  MLEs . There  must 
have  been  consistency  between  the  application  of  the  O&S 
cost  model  in  TLE  computations  and  MLE  computations.  Other- 
wise, comparisons  between  predicted  costs  (TLEs)  and  actual 
costs  (MLEs)  would  not  be  valid.  The  following  criteria 
were  used  to  determine  whether  or  not  consistency  existed 
in  the  areas  of  data  and  application  techniques: 

1.  The  same  components  (LRUs)  must  have  been 
evaluated  by  the  contractor  and  the  Air  Force. 
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2.  Data  for  the  computation  of  the  TLEs  and  MLEs 


had  to  be  sufficient  to  make  a valid  projection  of  O&S  costs 
for  the  LRUs  being  evaluated. 

3.  The  methods  used  to  develop  the  O&S  cost  pro- 
jection from  TLEs  had  to  be  the  same  methods  used  to  develop 
O&S  cost  predictions  from  MLEs. 

A related  study  on  the  application  of  the  O&S  cost 
model  states  that  the  LRUs  used  in  the  model  must  either  be 
the  same  or  must  account  for  the  total  O&S  costs  of  the 
entire  aircraft  (13:7-8).  This  was  the  basis  for  the  estab- 
lishment of  criterion  number  1.  All  data  necessary  to 
exercise  the  model  had  to  be  supplied  to  the  contractor  by 
the  Air  Force.  Failure  to  furnish  reliable  historical  esti- 
mates would  result  in  inaccurate  TLE  computations,  unless 
such  data  could  be  obtained  from  other  sources.  By  the 
same  token,  the  absence  of  reliable  A-10  maintenance  data 
would  result  in  inaccurate  MLE  computations. 

If  either  TLE  or  MLE  computations  were  of  question- 
able accuracy,  any  comparison  between  the  two  would  be 
inconclusive.  This  was  the  basis  for  criterion  number  2. 

Finally,  it  must  be  firmly  established  that  both 
Fairchild  and  the  Air  Force  applied  the  O&S  cost  model  in 
the  same  manner  (i.e.,  in  accordance  with  instructions  pro- 
vided in  the  SOW) . Any  errors  in  application  methods 
would  result  in  the  inability  to  make  a valid  comparison 
between  TLEs  and  MLEs.  Hence,  the  establishment  of 
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criterion  number  3.  Violation  of  any  one  of  these  criteria 
would  result  in  the  inability  to  make  a valid  TLE  to  MLE 
comparison.  Application  of  these  three  criteria  is  illus- 
trated in  Figure  3.  The  elements  of  this  investigation 
were  provided  via  interviews  with  the  contractors  and  by 
document  research. 

Accomplishment  of 
Objectives 

Previously  obtained  information  was  applied  to  the 
criteria  stated  below  for  determining  whether  or  not  Air 
Force  objectives  were  met.  Once  the  objectives  were  clearly 
established,  each  one  was  examined  separately  and  assessed 
according  to  these  criteria: 

1.  The  model  must  have  been  employed  toward 
accomplishing  this  objective. 

2.  The  data  and  techniques  used  in  applying  the 
O&S  cost  model  must  have  resulted  in  valid  or  accurate 
information  for  use  in  accomplishing  this  objective. 

Testing  criterion  1 for  each  objective  was  accom- 
plished through  discussion  with  the  A-10  SPO  supplemented  by 
document  research.  Criterion  2 was  tested  from  previous 
information  in  this  thesis  concerning  validity  of  the  com- 
parisons of  TLEs/MLEs . An  objective  was  considered  to  have 
been  fulfilled  only  if  both  criteria  were  met. 
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THEN: 

TLE/MLE  COMPARISON  -*  INVALID 

Where : 

F = Fairchild  DATA  = Data  used  for  O&S  cost 

AF  = Air  Force  predictions 

= Consistent  LRU  = Line  Replaceable  Units 

METHOD  = Method  of  Application 

Figure  3 

Comparison  of  Contractor  TLEs  and  Air  Force  MLEs 
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Interviews 

The  aforementioned  interviews  were  conducted  in  a 
nonstructured  fashion.  Answers  were  solicited  for  the 
questions  presented  in  Appendices  A,  B,  and  C. 
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Chapter  3 


AIR  FORCE  OBJECTIVES  IN  MODEL  APPLICATION 


Introduction 

This  chapter  addresses  the  question  of  what  the 
Air  Force's  objectives  were  in  using  an  O&S  cost  model  in 
the  A-X/A-10  acquisition  process.  The  information  provided 
herein  was  obtained  from  a number  of  sources  including 
personal  interviews  with  the  A-10  SPO  director  and  repre- 
sentatives from  Fairchild-Republic  Company  and  Northrop 
Corporation.  Determination  of  objectives  was  evaluated  only 
in  terms  of  common  agreement  and  clarity.  Whether  or  not 
these  objectives  were  met  will  be  addressed  in  Chapter  7 of 
this  thesis. 

Overall  Objective 

The  overall  objective  of  any  LCC  application  tech- 
nique is  to  minimize  the  overall  costs  of  owning  and  operat- 
ing a weapon  system.  Early  application  of  LCC  insures 
that  this  objective  will  be  considered  in  the  original 
design  of  the  system  (38).  An  indepth  study  of  Project  ABLE 
confirmed  that  this  was  the  primary  function  of  O&S  cost 
model  application.  A supplement  of  the  Project  states: 
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One  of  the  major  objectives  of  Project  ABLE  is  to 
cause  a new  system's  development  and  production  con- 
tractor (s)  to  give  careful  and  balanced  consideration  to 
logistics  implications  during  the  design  process  [17:2]. 

This  is  reiterated  in  another  supplement  of  the  same 

project:  "The  Project  ABLE  objective  is  not  to  forecast  true 

costs  but  to  create  the  appropriate  motivation,  neither  too 

large  or  too  small,  to  make  the  system  better  sooner  [16:2]." 

This  objective  was  transmitted  to  the  contractors  through 

the  Request  For  Proposal  (RFP) , F33657-70-R-0896 , a copy  of 

which  was  provided  each  competing  contractor.  A reference 

document  in  the  RFP  states  that: 

the  model  provides  the  contractor  a means  of  making 
design  and  management  tradeoffs  in  the  interest  of 
minimizing  (within  certain  constraints)  the  total  cost 
to  the  Government  of  A-X  ownershio  [2]. 

Thus,  the  Air  Force's  overall  objective  of  motivating  the 
contractors  to  design  a weapon  system  that  will  do  the  best 
job  with  the  least  overall  cost  to  own  and  operate  is  clear. 
It  has  also  been  established  that  this  objective  was  trans- 
mitted to  the  contractors.  The  question  still  remained  as 
to  the  contractor’s  understanding  of  this  objective. 

Mr.  D.  D.  Gregor,  one  of  Northrop 's  life  cycle 
costing  experts,  stated  that  "the  primary  objective  in  using 
the  O&S  cost  model  was  to  get  the  contractor  to  consider 
support  costs  during  the  design  phase  [14]."  A similar 
statement  was  made  by  Mr.  Robert  Thomas,  Fairchild's  leading 
O&S  model  expert.  He  stated  that  the  O&S  cost  model  was  to 
be  used  to  increase  the  reliability  and  maintainability 


within  design  to  cost  constraints  (29).  These  statements 
firmly  establish  that  the  contractors  understood  that  the 
O&S  cost  model  was  to  be  used  to  assess  support  costs  of 
the  A-X  aircraft  in  an  attempt  to  minimize  them. 


Objective  in  Source 
Selection 

The  next  objective  of  O&S  cost  model  application  to 
be  addressed  here  will  be  that  of  using  the  O&S  cost  model 
to  determine  which  of  the  competing  weapon  systems  exhibited 
the  least  O&S  costs  over  its  ten  year  life  cycle.  A supple- 
ment to  Project  ABLE  confirms  this  intent: 

Project  Able  makes  it  possible  to  include  within 
the  overall  source  selection  process  a meaningful 
approximation  of  the  total  life  cycle  cost  of  the 
systems  being  proposed  by  each  bidder  [18:vi]. 

Further  investigation  into  this  area  verifies  that  the 
contractors  were  informed  of  this  aspect  of  O&S  model  utili- 
zation. The  "Strawman"  model  that  was  used  to  validate  the 
O&S  model  states  that: 

The  contractors  have  been  advised  that  it  is  the 
intention  of  the  Air  Force  to  use  this  model,  or  some 
similar  model,  to  assess  the  supportability  of  the  pro- 
posed hardware,  and  further,  that  the  relative  support- 
ability  will  be  one  of  the  criteria  on  which  the  award 
of  the  production  contract  will  be  based  [5:2]. 

A copy  of  the  "Strawman"  model  was  given  to  each  competing 

contractor  in  June,  1970  (9:8). 

Here  again,  this  objective  has  been  established  by  the 
designers  of  Project  ABLE  and  transmitted  to  the  contractors 
through  documents  of  correspondence  (in  this  case,  the 


"Strawman"  model) . Contractor  understanding  of  this  objec- 
tive has  been  verified  by  documents  prepared  by  the  con- 
tractors: A Northrop  publication  states  that  one  of  the 
purposes  of  the  O&S  cost  estimate  was  to  "identify  the 
potential  operations  and  support  cost  of  the  A-9  [21:1]." 
Another  publication  obtained  from  the  Fairchild-Republic 
Company  states  that  "the  [O&S]  cost  model  provided  a stand- 
ardized and  precise  tool  for  application  to  the  A-X  source 

selection  preliminary  plan  dated  January,  1972  [9:3]." 

* 

A 

Thus,  the  objective  can  be  briefly  stated  as  follows:  The 
results  obtained  by  each  contractor  in  using  the  O&S  cost 
model  would  serve  as  a source  selection  criterion. 

Objective  in  Evaluating 
ECPs 

Investigation  revealed  that  a further  objective  in 
the  application  of  the  O&S  cost  model  to  tne  A-X/A-10 
acquisition  was  to  be  able  to  evaluate  the  impact  of  engi- 
neering change  proposals  (ECPs)  on  operation  and  support 
costs.  Colonel  J.  E.  Hildebrand,  in  a cover  letter  attached 
to  the  RFP , wrote:  "The  contractor's  best  efforts  are 
required  to  identify  the  preferred  designs  of  the  A-X  con- 
figuration items  . . . [and]  reduce  the  number  and  costs  of 
ECPs  [15]."  Additionally,  the  Statement  of  Work  clearly 
states  that  "the  model  will  be  used  to  determine  the 
logistics  effects  of  proposed  configuration  changes 
[3:5.3.12.2].”  Additional  documentation  of  this  objective 


can  be  found  in  the  Configuration  Management  section  of  the 
RFP: 


The  Government  evaluation  of  each  ECP  will  include 
a quantification  of  the  impact  of  the  proposed  change 
on  all  TLEs  ....  The  contractor  shall  not  implement 
an  ECP  until  such  action  has  been  approved  by  the 
Government  as  outlined  in  MIL  STD  480  . . . the  change 
order  approving  those  ECPs  adopted  will  reflect  the 
change  (if  any)  in  the  contractual  targets  for  all 
TLEs  (2:40]. 

Research  of  contractor  publications  has  verified  that  the 
contractors  were  well  aware  of  this  objective.  Documents 
prepared  by  both  contractors  referred  to  the  use  of  the  O&S 
cost  model  to  evaluate  ECPs  (9;  21). 


Objective  in  Award  Fee 
Determination 

The  final  objective  of  O&S  cost  application  to  the 
A-X  program  deals  with  the  awarding  of  an  incentive  fee. 
Project  ABLE  (16)  stipulates  that  TLEs  can  be  compared  to 
MLEs  collected  during  a validation  period  to  determine  the 
magnitude  of  an  award  or  penalty  to  assess  the  winning 
bidder  (16:7).  This  idea  was  presented  to  the  contractors 
in  the  RFP,  which  stated:  "The  award  fee  arrangement  shall 
be  a function  of  the  difference  between  TLE  and  the  value 
of  the  corresponding  MLE  [3]."  Documents  provided  by  the 
competing  contractors  (9;  21)  confirmed  that  they  realized 
the  O&S  cost  model  would  be  used  for  this  purpose.  The 
Production  Phase  Proposal  published  by  Northrop  Corporation 
alluded  to  this  objective  in  its  introduction: 
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[A]  purpose  of  this  O&S  cost  estimate  is  to  . . . 
establish  support  cost  objectives  that  USAF  can  use  to 
measure  the  success  of  the  contractor  in  designing 
for  minimum  support  requirements  [21:1]. 

A similar  article  prepared  by  Fairchild-Republic 
Company  discusses  the  use  of  the  O&S  cost  model  in  granting 
an  incentive  fee  (9). 

Summary 

Based  upon  the  research  findings  cited  above,  the 
objectives  of  the  application  of  the  O&S  cost  model  can  be 
summarized  as  follows: 

1.  To  instill  in  the  contractors  the  incentive  to 
consider  logistics  effects  (O&S  costs)  in  their  designs  of 
the  A-X  aircraft. 

2.  To  serve  as  a criterion  in  source  selection. 

3.  To  aid  in  the  evaluation  of  ECPs  with  respect 
to  O&S  costs. 

4.  To  serve  as  a criterion  in  the  granting  of  an 
award  fee. 

The  information  provided  above  has  verified  that  these 
objectives  were  submitted  to  the  contractors  through  the 
RFP,  SOW,  or  other  documents  provided  by  the  Air  Force. 
Further,  research  has  verified  that  the  contractors  received 
and  understood  these  objectives.  Subsequent  portions  of  this 
thesis  will  assess  the  methods  used  to  accomplish  these 
objectives  to  determine  whether  or  not  they  were  met. 
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Chapter  4 


AVAILABILITY  OF  DATA  FROM  D056 


Introduction 

This  chapter  examines  the  data  made  available  to 
Northrop  and  Fairchild-Republic  to  compute  their  Targeted 
Logistics  Effects  through  the  use  of  the  D056  Product 
Performance  Collection  System.  This  system,  a product  of 
AFM  66-1,  is  the  central  topic  of  this  analysis.  The 
reader  will  be  introduced  to  the  maintenance  data  philosophy 
expressed  in  Project  ABLE  and  subsequent  O&S  cost  model 
directives.  Following  this  is  an  overview  of  the  Air 
Force's  exercising  of  the  model  utilizing  F-4E  maintenance 
data  from  D056.  The  history  of  the  Air  Force's  test  use 
of  the  O&S  cost  model  will  then  be  compared  to  its  use  by 
the  two  prototype  contractors — Northrop  and  Fairchild- 
Republic. 

Maintenance  Data 
Philosophy 

Project  ABLE  (16),  developed  in  late  Spring  and 
early  Summer  of  1969,  is  the  foundation  document  of  the 
O&S  cost  model  (27).  Irving  Katz  (16;  17;  18;  19)  describes 
the  model  thusly: 

The  model  is  driven  by  the  characteristics  of  LRUs 
[Line  Replaceable  Units] . These  characteristics 
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include  RTS  [Repairable  This  Station] , NRTS  [Not 
Repairable  This  Station],  COND  [Condemnation]  rates, 
unit  prices,  repair  costs  . . . base  and  depot,  and 
MTBF  [Mean  Time  Between  Failure]  [16: B—  7 ] . 

The  TLEs  driven  by  LRUs  are  TLE^,  initial  and  replenishment 
spares;  TLE2,  off-equipment  maintenance — excluding  engines ; 
and  TLE^ , on-equipment  maintenance  (7;  9;  16).  The  contri- 
butions of  TLEs  1-3  to  the  total  predicted  O&S  costs  cannot 
be  overstated.  Seventy-one  percent  of  Fairchild's  total 
TLE  was  accounted  for  by  these  three  equations  (9:13). 
Further,  sixty-eight  percent  of  the  total  MLE  computed  by 
the  United  States  Air  Force  was  found  in  these  maintenance- 
related  elements  (9:23).  Fuel  costs,  which  contributed 
seventeen  percent’  to  the  total  MLE,  were  accounted  for  as 
prescribed  in  AFR  173-9,  Aircraft  Fuel  Consumption  Data 
Collection,  and  are  not  addressed  in  this  study.  What, 
then,  is  on-equipment  and  off-equipment  maintenance? 

On-equipment  maintenance  [as  defined  in  AFM  66-1, 

11  Mar  68]  . . . includes  servicing,  preventive  mainte- 
nance (including  preflights,  postflights,  and  look 
phase [s]  of  scheduled  organizational  maintenance),  time 
change  removals  and  unscheduled  removals  [18:44]. 

Off-equipment  cost  figures  are  developed  around  the  "average 
value  of  labor"  where  " [the]  average  cost  to  repair  includes 
the  average  value  of  labor — salary,  fringe  benefits,  over- 
head, and  lost  time — utilized  in  the  repair  process  [18:44]." 
The  RFP  (2;  3)  instructs  the  contractors  to  apply  these  labor 
cost  figures  to  "the  repair  of  [all]  LRUs  removed  from  the 
force  over  the  10  year  life  cycle  [of  the  air  vehicle]  [2]." 


In  the  Request  for  Proposal  (RFP)  transmitted  to 
Northrop  and  Fairchi Id-Republic , the  United  States  Air 
Force's  philosophy  on  TLE  computation  was  related  as 
follows:  "MTBR  [Mean  Time  Between  Removals]  may  be  best 

supported  by  a comparison  of  the  [proposed]  hardware  to  hard- 
ware currently  operational  [in  Air  Force  systems]  [3]." 

Data  elements  provided  by  AFM  66-1/D056  will  include: 

(1)  unit  prices  of  LRUs,  (2)  depot  costs,  and  (3)  costs  of 
maintenance  consumables  (2) . Further,  "AFM  66-1  data  will 
be  used  [during]  the  test/validation  period  [2]."  A study 
completed  by  the  United  States  Air  Force  Academy  (7)  further 
embraces  the  use  of  AFM  66-1  maintenance  data:  "Necessary 
data  for  the  [O&S]  model  comes  from  AFLC  66-1  data  compiled 
. . . as  part  of  routine  Air  Force  K0512  computer  runs 
[7:2-3] 

Air  Force  Exercise 
of  Model 

On  4 November  1970,  Lt  General  Gideon,  the  AFLC 
Vice  Commander,  charged  his  Operations  Analysis  Office  with 
exercising  the  Project  ABLE  model  with  "the  necessary  and 
appropriate  data  from  within  existing  Air  Force  data  systems 
[5:1]."  In  Technical  Memorandum  Number  14,  published  by  the 
Operations  Analysis  Office,  Headquarters,  Air  Force  Logistics 
Command,  the  data  base  was  defined  as  follows:  "The  bulk 


2K051  provides  maintenance  data  inputs  to  D056. 
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of  the  data  used  for  the  exercise  was  AFM  66-1  data  as 
processed  by  the  D056  data  [collection]  system  [5:1]." 

MLEs  1-3  were  computed  from  AFM  66-1  data  gathered  during 
the  simulated  test  period  (5:5).  Captain  Raymond  Cavender 
(5;  6),  in  his  report  Application  of  the  A-X  10-Year  Operat- 
ing and  Support  Cost  Model  to  the  F-4E,  provides  the 
following  insights  to  the  problems  encountered  in  computing 
on-equipment  and  off-equipment  cost  figures: 

In  the  first  batch  of  data  we  obtained  from  the  0056 
system,  information  had  been  collected  by  Work  Unit 
Codes  (WUCs) ...  it  was  our  intention  to  consider  the 
WUCs  as  the  identifiers  of  the  line  items  in  our  compu- 
tation. This  approach  soon  got  us  into  a great  deal 
of  trouble,  resulting  mainly  from  the  fact  [that]  there 
were  usually  several  Federal  Stock  Numbers  (FSNs) 
reported  against  a single  WUC  ....  As  we  got  deeper 
into  this  problem,  we  realized  that  we  could  not  use 
WUCs  as  identifiers  of  separate  line  items.  The  only 
way  to  extricate  ourselves  from  the  problem  was  to 
retrace  our  steps  and  gather  data  by  FSN  [5:6]. 

Work  unit  coded  data  was  retained  for  use  in  the  computa- 
tions of  on-equipment  cost  figures  as  these  man-hours  are 
collected  by  WUC,  not  FSN  (5).’  Unfortunately,  the  off- 
equipment  calculations  were  further  hampered  by  the  absence 
of  depot  level  repair  data  in  D056.  Captain  Cavender  con- 
tinues : 


We  lost  track  of  an  item  once  it  went  to  the  depot 
for  repair  ....  As  a result  of  this  situation,  we 
didn't  have  the  data  on  [the]  cost  to  repair  individual 
failed  items  at  [the]  depot  ....  [To  estimate  costs] 
we  fell  back  on  the  exchange  rates  from  the  Depot  Mainte- 
nance Industrial  Fund  (DMIF) 3 ....  [If]  the 


3These  exchange  rates  are  computed  by  stock  class 
and  not  by  line  item. 
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exchange  rate  was  not  computed,  a world-wide  average 
of  20%  was  used  [5:10]. 

Despite  these  problems,  the  study  (5)  concluded  that: 

This  undertaking  has  indicated  that  the  A-X  Operat- 
ing and  Support  Cost  model  can  be  meaningfully  exercised 
using  data  generally  available  within  current  Air  Force 
data  systems  [5:9]. 


Contractor  Use  of  O&S 
Cost  Model 


The  remaining  portion  of  this  chapter  will  address 
the  guidance  provided  by  the  United  States  Air  Force  to 
Northrop  and  Fairchild-Republic  concerning  TLE  computations 
and  contractor  experience  in  O&S  model  application. 

The  RFP  transmitted  to  the  prototype  contractors 
provided  the  following  guidance  for  LRU  selection: 

A comparative  analysis  to  hardware  currently  in 
military  or  commercial  use  will  be  considered  suitable 
. . . detailed  information  will  be  input  [to  the  Air 
Force]  to  the  extent  deemed  feasible,  and  practical, 
to  support  the  [repair]  man-hour  values  claimed  [2] . 

The  primary  United  States  Air  Force  provided  data  source 

was  presented  thusly:  "The  competing  contractors  on  the  A-X 

program  [will]  have  access  to  large  amounts  of  [maintenance] 

data,  primarily  through  the  [AFM]  66-1  data  system  . . . 

[6:5]."  Air  Force  directives  further  stated  labor  rate 

estimates  included  in  TLE2  and  TLE^  were  to  include  salary, 

fringe  benefits,  overhead,  lost  time,  and  consumable 

material  for  E-3,  E-5,  and  E-7  skill  levels  (3).  LRU 

maintenance  was  to  be: 

. . . tracked  through  their  repair  or  condemnation 
cycle  through  local  [and/or]  specialized  repair 


activities.  The  data  . . . will  be  indexed  by  Manu- 
facturer's Part  Number  and  [federal]  stock  number  [2], 


The  necessity  for  the  contractors  to  have  complete  data 
with  which  to  make  their  TLE  computations  was  expressed  by 
the  Joint  AFSC/AFLC  Commanders'  Working  Group  on  Life  Cycle 
Costing  (35) : 

The  A-X/A-10  0 and  S cost  model  . . . require  [s] 
input  parameters  which  describe  support  requirements 
such  as  MTBR,  base  and  depot  repair  costs  and  condem- 
nation rates  . . . the  ability  of  the  contractor [s]  to 
acquire  complete  and  valid  data  in  a timely  manner 
must  be  assessed  at  the  time  the  model  is  definitized 
and  its  use  specified  [5:5,10]. 

The  relative  value  of  comprehensive  maintenance  data  was 

expressed  by  Northrop  Corporation  in  their  "A-9  Operational 

and  Support  (O&S)  Cost  Summary": 

Maintenance  analysis  of  USAF  tactical  aircraft 
and  . . . [their]  maintenance  significant  components 
show  15%  of  the  maintenance  components  accounted  for 
80%  of  the  air  vehicle  maintenance  [21:8]. 

Mr.  Robert  Thomas  (29;  30)  further  echoes  this  point  in  his 

"Data  Requirements  for  05.S  Cost  Analysis — An  Industry  Point 

of  View" : 

Unless  valid  data  is  available  and  utilized,  it  is 
unlikely  [that]  there  will  be  any  resemblance  between 
actual  operating  and  support  costs  of  a weapons  system 
and  that  predicted.  Thus  time,  effort,  and  money 
wasted  [30] . 

When  the  contractors  began  to  exercise  the  model, 

_ computing  TLE2  and  TLE^  became  an  unmanageable  problem.  A 
Fairchild  publication  states: 

Considerable  confusion  has  existed  on  chargeable 
and  non-chargeable  maintenance  hours  for  the  prediction 
...  of  on-equipment  and  off-equipment  maintenance  . . . . 
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Depot  maintenance  data  was  probably  the  biggest  problem 
area  encountered  in  obtaining  data  for  the  A-10  cost 
model  exercise.  Various  channels  were  exercised  in 
our  efforts  to  obtain  depot  data  . . . and  the  results 
received  were  very  disheartening  [30:v-6]. 

Mr.  Herbert  F.  Harris,  head  of  Northrop  Corporation's  Cost 

Analysis  Branch  during  the  A-X  competition,  supported 

Fairchild's  position: 

Depot  data — 66-1 — was  hard  to  get  because  it  was 
considered  proprietary  information.  The  66-1  system 
seems  difficult  to  manage  as  its  reliability  as  a data 
base  is  questionable.  Field  data  needs  to  be  more 
reliable  and  controlled  [14]  . 

Fairchild,  when  requesting  relief  from  this  dilemma,  stated: 

The  problem  with  utilizing  the  66-1  Maintenance 
Data  Collection  System  is  the  incompatability  between 
the  maintenance  [man-hour]  data  collected  and  the 
chargeable  maintenance  man-hours  predicted  by  the  con- 
tractor [40]. 

The  United  States  Air  Force's  view  of  this  problem  was 
expressed  by  the  Joint  AFSC/AFLC  Commander's  Working  Group  on 
Life  Cycle  Costing: 

The  lack  of  preparation  of  a data  retrieval  system 
to  support  the  A-X  O&S  cost  model  input  requirements 
placed  an  unacceptable  burden  upon  the  Deputy  Program 
Manager  for  Logistics  and  [the]  contractor  logistics 
analysts  when  the  data  had  to  be  retrieved  on  existing 
systems  [AFM  66-1/D056]  for  use  in  estimating  A-X/A-10 
model  input  values  ....  [The]  unfamiliarity  with 
various  weapon  systems,  the  maintenance  data  collection 
system,  and  the  various  coding  techniques  used  by  the 
Air  Force  caused  a large  expenditure  of  contractor 
resources  [35:11]. 

The  final  problem  in  data  completeness  and  usability  arose 
when  the  United  States  Air  Force  evaluated  the  prototype 
contractor's  TLE  proposals. 


Since  there  was  a limited  time  for  proposal  evalua- 
tion, the  AFLC  data  bank  could  not  be  used  for  several 
reasons.  [The  contractors]  gave  Aeronautical  Equipment 
Reference  Numbers  (AERNO)  for  some  LRUs  and  Federal 
Stock  Numbers  (FSN)  for  others,  the  AFLC  data  bank  is 
keyed  primarily  on  Work  Unit  Codes  (WUCs) . The  pro- 
cedure to  go  from  AERNO  or  FSN  to  WUC  is  not  simple, 
and  often  the  conversion  was  impossible  because  no 
acceptable  cross-references  were  available  in  AFLC 
[35:11-12] . 


Summary 

This  chapter  has  introduced  the  reader  to  the  D056 
Maintenance  Data  Collection  System  and  its  contribution  to 
the  first  three  TLE  formulae.  A strong  parallel  exists 
among  those  maintenance  data  problems  encountered  in  the 
first  exercise  of  the  model  with  F--4E  D056  data,  the  con- 
tractors’ TLE  costing  estimates,  and  the  proposal  evaluation 
board’s  attempts  to  judge  each  contractor's  TLE.  The  lack 
of  resolution  of  maintenance  man-hour  data  problems,  com- 
bined with  no  method  of  cross-referencing  AERNO,  FSN,  and/or 
WUC  data  leads  to  the  conclusion  that  the  model  cannot  be 
exercised  by  data  obtained  from  the  D056  Maintenance  Data 
Collection  System,  as  used  in  the  A-X/A-10  O&S  cost  model. 
Appendix  D provides  an  overview  of  the  AFM  66-1  Maintenance 
Data  Collection  System  and  its  outputs.  Alternatives  will 
be  discussed  in  the  final  chapter  of  this  thesis. 
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Chapter  5 


VALIDITY  OF  COMPARISONS  BETWEEN 
CONTRACTOR  TLEs 


Introduction 

Target  Logistics  Effects  served  as  key  variables  in 
the  application  of  the  O&S  cost  model  by  allowing  each  con- 
tractor to  project  his  success  in  reducing  costs  associated 
with  reliability  and  maintainability.  TLEs  were  used 
initially  as  part  of  the  source  selection  process  and  later 
for  the  structuring  of  an  incentive  contract  (19:8).  This 
segment  of  the  research  will  examine  the  techniques  used  by 
Fairchild-Republic  Company  and  Northrop  Corporation  to  cal- 
culate TLEs.  The  purpose  here  is  to  determine  if  the  data 
and  methods  used  to  make  these  calculations  were  consistent 
between  the  contractors.  Criteria  established  in  the 
Methodology  for  this  evaluation  will  be  restated  here: 

1 . The  same  data  base  must  have  been  used  by 
Northrop  and  Fairchild  for  TLE  computations  (historical  data 
was  to  have  been  provided  by  the  Air  Force) . 

2.  The  same  criteria  must  have  been  used  by  both 
contractors  in  the  selection  of  LRUs. 

The  data  provided  by  the  Air  Force  and  the  techniques  used 
for  LRU  selection  will  be  analyzed  in  order  to  determine  if 
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a valid  comparison  can  be  made  in  accordance  with  Figure  2, 
page  16. 


Data  Base 

To  test  the  first  criterion,  contractor  representa- 
tives were  asked  what  data  was  provided  them  by  the  Air 
Force  to  exercise  TLE  computations.  These  computations 
were  made  from  historical  data  collected  on  weapons  systems 
currently  in  the  USAF  inventory  (14;  29).  According  to 
Northrop' s A-9  Production  Phase  Proposal,  "USAF  maintenance 
data  for  the  F-4E,  A-7D,  F-105,  F-5,  F-100,  A-37,  and  F-104 
were  used  to  relate  A-9  maintenance  requirements  with  cur- 
rently operational  hardware  [21:4]."  Fairchild  also 
received  maintenance  data  on  inventory  aircraft  (30:5). 

Much  of  this  data  was  provided  to  both  contractors  through 
the  Maintenance  Data  Collection  System  specified  in  AFM  66-1 
(24).  An  analysis  of  this  data  is  provided  in  Chapter  4 of 
this  thesis. 

The  information  presented  above  supports  the  fact 
that  both  contractors  were  furnished  with  maintenance  infor- 
mation from  aircraft  currently  in  the  USAF  inventory  and  that 
both  contractors  were  provided  equal  access  to  the  AFM  66-1 
Maintenance  Data  Collection  System. 

LRU  Selection 

Questions  were  also  addressed  to  the  contractors' 
representatives  regarding  the  process  used  in  selecting  LRUs 
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to  be  assessed  by  the  model.  Documents  were  provided  which 
explained  the  methods  used  to  make  these  selections.  The 
process  used  by  Northrop  in  LRU  selection  is  explained  in 
the  Production  Phase  Proposal: 

Maintenance  analysis  of  USAF  tactical  aircraft  and 
identification  of  maintenance  significant  components 
shows  that  15  percent  of  the  maintenance  components 
account  for  80  percent  of  air  vehicle  maintenance.  This 
assessment  of  A-9  maintenance  significant  components  was 
used  to  establish  A-9  maintenance  significant  line 
replaceable  units.  The  A-9  OSS  LRUs  include  the  hi-value 
hardware  and  those  maintenance  significant  LRUs  for 
which  more  than  three  removals  are  expected  during  the 
aircraft's  design  life.  An  initial  list  comprised  of 
approximately  250  components  has  been  compiled  for  par- 
ticular emphasis  [21:2,8-9]. 

This  statement  supports  the  fact  that  Northrop  Corporation 
selected  LRUs  based  upon  items  that  were  historically  high- 
dollar  maintenance  items. 

A study  prepared  by  Fairchild-Republic  Company 
describes  their  process  used  for  LRU  selection  as  follows: 

Forms  were  prepared,  current  and  historical  mainte- 
nance data  was  ordered,  extracted  and  assembled  and  the 
maintenance  prediction  values  for  the  parameters 
peculiar  to  this  requirement  were  prepared  by  the  system 
maintenance/maintainability  engineer  [s]  ....  The 
extensive  military  aircraft  experience  of  the  group 
enabled  them  to  quickly  identify  over  500  LRUs  and  to 
select  the  most  comparable  in-service  item  for  compari- 
son [9:6]. 

Although  no  mention  is  made  here  of  high  O&S  cost  items,  an 
interview  with  Fairchild-Republic ' s O&S  cost  expert  indi- 
cated that  this  factor  was  considered  in  LRU  selection  (29). 

It  would  appear  that  both  companies  based  LRU 
selection  on  the  criterion  of  high  O&S  cost.  The  disparity 
in  the  number  of  LRUs  selected  by  each  company,  however,  led 


to  further  research  in  this  area.  It  was  determined  that 
problems  were  encountered  by  both  contractors  in  the  selec- 
tion of  LRUs.  One  such  problem  was  the  definition  of  the 
term  LRU.  This  is  expounded  on  in  a paper  written  by 
Robert  Thomas  of  Fairchild- Republic  Company.  He  states: 

The  definition  of  an  LRU  . . . caused  concern  in 
the  designation  of  LRUs  to  be  used  in  the  cost 
model  ....  Almost  any  part  of  an  aircraft  can  be 
removed  and  replaced  on  the  flight  line  ....  Also, 
in  an  effort  to  simplify  the  O&S  costing  exercise,  cal- 
culations were  made  on  what  was  considered  as  signifi- 
cant and  cost  sensitive  LRUs.  However,  by  definition 
in  the  cost  model,  even  minor,  low  cost  insensitive 
items  are  LRUs  [30:5]. 

These  views  were  also  expressed  by  Northrop  personnel 
(13;  14) . There  was  no  guidance  given  by  the  Air  Force 
concerning  whac  LRUs  should  be  examined  and  what  should  be 
deemed  as  high  cost  drivers  (14).  A study  prepared  by  the 
Joint  AFSC/AFLC  Commanders'  Working  Group  on  Life  Cycle  Costing 
verifies  the  lack  of  Air  Force  participation  in  the  LRU 
selection  process: 

Prior  to  the  source  selection  period  those  LRUs 
comprising  either  the  A-9  or  the  A-10  weapons  systems 
were  unknown  until  the  proposals  were  received  by  the 
evaluation  board  [35:11]. 

Results  of  the  research  cited  above  support  the 
fact  that  LRU  selection  was  done  by  each  contractor  sepa- 
rately, with  minimum  coordination  with  the  Air  Force.  While 
attempts  were  made  by  both  contractors  to  select  high  O&S 
cost  drivers  as  LRUs,  no  agreement  was  reached  on  the 
definition  of  the  term  "high  cost"  or,  for  that  matter,  the 
definition  of  "LRU"  (14;  29). 
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Summary  and 
Conclusions 

To  determine  if  a valid  comparison  between  contractor 
TLEs  can  be  made,  two  criteria  have  been  tested  here.  The 
first  addresses  the  data  base  used  by  the  contractors. 
Information  cited  above  supports  the  fact  that  both  contrac- 
tors were  provided  with  historical  maintenance  data  from 
operational  USAF  aircraft  and  both  contractors  were  given 
equal  access  to  the  USAF  Maintenance  Data  Collection  System. 
This  is  sufficient  to  support  achievement  of  criterion 
number  1. 

Criterion  number  2 deals  with  the  criteria  used  by 
each  contractor  to  select  LRUs.  It  has  been  determined  that 
there  was  a lack  of  coordination  between  contractors 
coupled  with  the  absence  of  firm  definitions  by  the  USAF  of 
the  terms  'LRU'  and  ’high  cost*.  Contractor  determination 
of  LRUs  was  done  independently  using  different  criteria. 
Therefore,  criterion  number  2 is  violated.  Thus,  any  com- 
parison between  contractor  TLEs  regarding  OfcS  cost  would 
not  be  valid.  This  is  illustrated  in  Figure  4. 


TLE  COMPARISON  INVALID 


Where: 

F = Fairchild 
N = Northrop 
~ = Consistent 
DATA  = Data  Base 

LRU  DETERMINATION  = Criteria  used  to  identify  LRUs 


Chapter  6 


VALIDITY  OF  COMPARISONS  BETWEEN 
TLES  AND  MLEs 


Introduction 

As  stated  in  Chapter  3,  one  of  the  objectives  of 
O&S  cost  model  applications  in  the  A-X/A-10  acquisition  was 
to  use  the  model  as  a device  for  determining  the  magnitude 
of  an  award  fee.  Irving  Katz  refers  to  this  application  in 
Project  ABLE  where  he  states: 

The  difference  between  MLE  and  the  TLE  of  the 
winning  bidder  is  treated  by  a predetermined  arrange- 
ment through  which  the  government  and  the  contractor 
share  in  the  extra  benefits  or  extra  penalties  [17:7]. 

In  order  to  compare  TLEs  and  MLEs,  "there  must  be  complete 

compatibility  between  them  in  terms  of  scope,  timing  and 

ground  rules  [7:3]."  The  purpose  of  this  chapter  is  to 

determine  if  such  compatibility  existed  between  Fairchild’s 

TLE  projections  and  the  Air  Force's  MLE  projections.  The 

criteria  by  which  this  assessment  has  been  made  are  listed 

in  the  Methodology  and  will  be  restated  now: 

1.  The  same  components  (LRUs)  must  have  *b*een 
evaluated  by  the  contractor  and  the  Air  Force. 

2.  Data  for  the  computation  of  TLEs  and  MLEs  had  to 
be  sufficient  to  make  a valid  projection  of  O&S  costs 

for  the  LRUs  being  evaluated. 


3.  The  methods  used  to  develop  the  projec- 
tion from  TLEs  had  to  be  the  same  methods  used  to  develop 
the  10  year  projection  from  MLEs. 

The  first  of  these  three  criteria  can  be  associated  with 
scope,  and  the  last  two  with  timing  and  ground  rules. 

Components  Evaluated 

The  first  criterion  was  the  simplest  to  test.  The 

intention  to  evaluate  the  same  LRUs  when  computing  TLEs 

and  MLEs  was  suggested  in  the  "Strawman"  model: 

The  winner  of  the  contract  is  contractually  bound 
to  achieve  in  the  test  period  a composite  of  test 
values  such  that  when  they  are  put  through  the  same 
model,  a measured  logistics  effect  as  least  as  good  i.e. 
(as  low)  as  the  TLE  will  result  [5:4]. 

Further  documentation  provided  support  for  the  fact  that 
the  LRUs  evaluated  were  to  be  the  same  for  TLEs  and  MLEs. 

The  RFP  states  that  analogues  for  TLEs  shall  be  denoted 
MLEs  and  the  same  LRUs  are  specified  (2) . A United  States 
Air  Force  Academy  study  on  the  O&S  cost  model  application 
states  that  "MLEs  are  exactly  analogous  to  TLEs,  except  that 
MLEs  use  operational  data  instead  of  estimates  based  on 
historical  experience  as  in  TLEs  [7:4]."  Interviews  with 
contractor  representatives  and  SPO  representatives  confirmed 
that  the  list  of  LRUs  selected  by  the  contractor  was  sub- 
mitted to  the  SPO  and  that  these  LRUs  were  evaluated  for  MLE 
computations  (24;  29).  Based  on  the  foregoing  information, 
TLEs  and  MLEs  were  compatible  in  terms  of  scope. 
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Sufficiency  of  Data 

Data  availability  via  the  D056  has  previously  been 
addressed  in  Chapter  4.  In  this  section,  we  will  examine 
sufficiency  of  the  data  used  in  actual  O&S  cost  computations 
for  TLEs  and  MLEs.  The  ability  of  the  contractor  to  obtain 
sufficient  data,  despite  the  problems  encountered  in  using 
D056  data,  will  be  examined  now. 

Contractor  data.  Several  specific  problem  areas  have  been 

pointed  out  by  the  Fairchild-Republic  Company.  In  a report 

published  by  that  company,  the  following  statement  appears: 

Data  for  WUC  01  Ground  Handling,  Servicing  and 
Related  Tasks,  is  not  available  to  contractors  for 
predictive  analysis  or  contractor  verification.  Since 
WUC  01  MLE  for  the  A-10  was  25%  of  the  total  on- 
equipment  maintenance  [MLE^]  which  in  turn  was  53.5%  of 
the  total  MLE,  it  can  readily  be  seen  that  this  is  a 
very  significant  maintenance  cost  [9:12]. 

Similar  problems  were  incurred  with  Depot  Repair  man-hour 
data.  This  data  was  not  available  to  the  contractor  because 
it  is  competition  sensitive  (14)..  The  66-1  MDC  system  does 
not  report  this  data  (9:12). 

Another  article  written  by  Mr.  Robert  Thomas  of 
Fairchild  states  that  no  definition  of  chargeable  mainte- 
nance man-hours  was  presented  to  the  contractor.  The 
problem  was  whether  to  consider  only  productive  direct 
maintenance  costs  or  all  inclusive  maintenance  costs  reported 
in  the  66-1  Maintenance  Data  Collection  System  (30:5,  7). 


what  costs  to  exclude  for  labor  mentioned  above,  and  other 
costs,  was  not  resolved.  Fairchild-Republic  summarizes 
this  problem  by  stating  that: 

More  definition  is  required  here  to  specify  what  is 
to  be  included  and  excluded,  for  example,  Weapons 
Loading,  and  more  information  is  required  as  to  availa- 
bility of  sources  of  comparative  data  for  predictive 
analysis  in  this  and  other  support  general  categories 
19:12]. 

If  Fairchild  included  only  direct  labor  costs  in  its  TLE 
calculations  and  the  Air  Force  included  both  direct  and 
indirect  labor  costs,  the  ability  to  make  a valid  comparison 
between  TLEs  and  MLEs  would  be  significantly  impaired.  Lack 
of  clear  definitions  for  certain  data  inputs  required  to 
exercise  the  model  may  have  led  the  contractor  to  include 
some  costs  that  were  excluded  by  the  Air  Force  and  vice 
versa . 

USAF  data.  Having  examined  the  data  input  problems  related 
to  contractor  TLE  computations,  an  analogous  study  will  now 
be  reviewed  regarding  Air  Force  MLE  computations.  Computa- 
tions of  MLEs  1-4  were  computer  augmented  with  K051  data4 
and  fed  into  this  program  automatically  (7:4).  This  investi- 
gation dealt  only  with  calculations  of  MLEs  1-4.  The  MLE 
projections  used  by  the  Award  Fee  Determination  Board  were 
derived  from  5,287  hours  of  flying  time  (9:10).  According 
to  one  United  States  Air  Force  Academy  (USAFA)  study: 

“A  subsystem  of  D056. 
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The  5000  hour  timeframe  is  inadequate  for  several 
reasons.  By  the  time  5000  hours  of  flying  time  has 
been  accumulated  on  the  airframe,  many  of  the  LRU 
prices  are  still  not  available.  The  data  itself  is 
incomplete  at  this  point  because  it  lacks  information 
on  removal  rates  of  numerous  LRUs.  Some  LRUs  are  just 
not  removed  before  5000  hours  [7:9]. 

Further  investigation  was  conducted  in  this  area  to 
determine  the  methods  of  arriving  at  and  the  accuracy  of 
unit  prices.  This  investigation  revealed  that  approximately 
sixty-five  percent  of  the  unit  prices  of  the  LRUs  had  to 
be  estimated  at  the  5,287  hour  point.  These  estimates  were, 
for  the  most  part,  obtained  from  conferring  with  engineering 
personnel  who  were  familiar  with  prices  of  comparable  sys- 
tems in  the  USAF  inventory  (11).  To  determine  the  accuracy 
of  these  estimates,  twenty  LRUs  were  selected  at  random  and 
actual  prices  for  these  LRUs  were  obtained  from  the  item 
managers  at  their  respective  ALCs.  Of  these  twenty  items, 
thirteen  had  been  replaced  by  new  FSNs  and  prices,  which  did 
not  appear  in  any  of  the  computer  products.  Of  the  seven 
remaining  items,  two  were  correct  and  the  other  five  were 
overstated.  Results  show  that  the  estimated  prices  were 
overestimated  by  an  average  of  forty  percent.  This  clearly 
indicates  that  the  5000  hour  point,  because  of  insufficient 
data  on  actual  unit  prices,  was  inadequate  for  accurate 
MLE  computations. 

An  analysis  was  conducted  to  determine  if  there  was 
a significant  difference  in  LRU  removal  rates  between  the 


5000  hour  point  and  the  25,000  hour  point.  In  some 
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instances,  removal  rates  could  not  be  tracked  on  a cumula- 
tive basis  at  any  point  in  time.  A recent  study  conducted 
by  the  USAFA  (32),  in  October  1977,  addressed  this  problem. 
This  study  stated  that  test  removals  were  not  being  properly 
aggregated  in  the  computer  runs.  Mean  Times  Between 
Removal  (MTBR)  were  being  computed  using  aggregated  flying 
hours  and  unaggregated  test  removals  (TR)  (32).  The  SPO 
initiated  action  for  changes  to  be  made  in  the  program  in 
an  attempt  to  correct  this  prob  (24;  31).  It  was  sub- 
sequently determined  that  data  was  not  aggregated  in  the 
program  because  the  AFM  66-1  system  does  not  retain  the  data 
for  a long  enough  period  (36;  37) . "Even  if  the  program  had 
the  capability  to  aggregate  test  removals,  it  would  not  have 
had  the  data  to  do  so  [11]."  Since  MTBRs  and  aggregated 
test  removals  are  necessary  inputs  for  the  O&S  cost  model, 
inaccuracies  in  these  figures  would  result  in  inaccuracies 
in  the  final  MLE  figures,  rendering  any  comparison  between 
TLEs  and  MLEs  questionable  at  best. 


Methods  of  Application 


The  methods  used  to  calculate  TLEs  and  MLEs  were 


examined  to  determine  if  any  aspects  of  model  application 
adversely  affected  the  comparison  made  between  them. 

One  important  aspect  considered  was  the  time  period 
over  which  the  TLEs  and  MLEs  were  calculated.  Katz  stated 


the  importance  of  using  the  same  time  period  for  TLE  and 
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MLF  calculations  in  the  ABLE  study  (16:3).  Using  the  same 
time  period  for  evaluation  of  TLEs  and  MLEs  is  important 
because  of  the  difference  in  O&S  costs  that  are  experienced 
throughout  the  life  cycle  of  a weapons  system  (7:5).  This 
difference  is  demonstrated  by  the  "bathtub"  curve  in 
Figure  5 which  is  widely  accepted  as  typical  of  O&S  costs 
for  most  weapons  systems  (22:178).  The  Incentive  Award 
Provisions  of  the  Statement  of  Work  (SOW)  specify  that  the 
analysis  of  the  differences  between  the  TLEs  and  MLEs  shall 
be  for  the  same  time  period  (3). 


Figure  5 

Failure  Rate  Versus  Time  [22:178] 
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According  to  a Fairchild  report,  the  Air  Force 
instituted  a change  in  the  measurement  period  after  the 
contract  was  awarded  and  "final  agreements  were  reached 
relative  to  the  change  in  TLE  value  to  accord  with  the  change 
in  the  measurement  period  [9:10]."  Based  upon  this  infor- 
mation, TLEs  and  MLEs  were  compatible  in  terms  of  timing. 

Further  investigation  was  conducted  in  other  areas 
of  model  application.  Documents  verify  that  the  contractor 
was  provided  with  specific  instructions  regarding  the  appli- 
cation of  the  O&S  cost  model  (2;  3;  9;  10;  29).  An  entire 
annex  is  dedicated  to  this  subject  in  the  RFP  (3) . An 
interview  with  one  of  Fairchild's  O&S  cost  model  experts 
indicates  that  the  coordination  with  the  Air  Force  concern- 
ing model  application  was  excellent  (29) . Further,  Fairchild 
personnel  demonstrated  an  excellent  working  knowledge  of 
the  model  which  lends  credence  to  the  assumption  that  the 
theory  behind  the  model  and  the  mechanics  therein  were  fully 
understood  by  those  at  Fairchild  responsible  for  its  appli- 
cation (29).  It  should,  however,  be  pointed  out  that  actual 
calculations  were  not  available  for  examination. 

Air  Force  calculations,  on  the  other  hand,  are 
available  and  were  examined.  Under  the  guidance  of  the 
SPO,  the  formulae  were  applied  using  the  5,287  hour  data  and 
it  was  determined  that  these  formulae  were  properly  exercised 
by  the  computer  (29).  Problems  were  discovered  in  the 
computation  of  MLE^ . The  5,287  hour  O&S  cost  model  computer 


product  for  MLEs  1-4  shows  the  total  of  MLE^  to  be  approxi- 
mately 159  million  dollars,  while  the  final  report  used  by 
the  Evaluation  Board  shows  this  figure  to  be  78.4  million 
dollars.  In  discussing  this  discrepancy  with  the  SPO,  it 
was  determined  that  the  computer  program  was  overstating 
MLE^  due  to  problems  in  the  1300 5 WUC  area.  In  this  area, 
several  WUCs  exhibited  extremely  high  unit  prices.  Addi- 
tionally, test  removals  were  reported  for  items  such  as 
wheels  and  brake  assemblies  when  only  tire  removals  should 
have  been  reported.  Manual  alterations  were  administered  in 
order  to  correct  these  discrepancies  (24).  This  accounts 
for  the  difference  in  the  computed  value  of  MLE^  and  the 
reported  value.  Another  manual  change  was  the  addition  of 
support  general  cost  data  to  the  computed  value  of  MLEs  (1) . 
It  was  not  possible  to  track  the  accuracy  of  these  manual 
changes.  Therefore,  no  firm  conclusions  can  be  made  con- 
cerning the  compatibility  of  Air  Force  and  contractor  appli- 
cation techniques. 

Summary  and  Conclusions 

As  stated  earlier  in  this  chapter,  a valid  compari- 
son between  TLEs  and  MLEs  requires  complete  compatibility  in 
terms  of  scope,  timing,  and  ground  rules.  Thus,  only  if  the 
same  LRUs  were  evaluated  by  both  the  Air  Force  and  the 

’’wheels,  tires  and  brakes. 


contractor,  and  sufficient  data  was  available  for  10-year 
O&S  cost  projections,  and  the  methods  used  to  develop  TLEs 
and  MLEs  were  the  same,  could  a valid  comparison  be  made. 

The  findings  discussed  in  this  chapter  reveal  that 
although  the  same  LRUs  were  evaluated  by  both  Fairchild  and 
the  Air  Force,  data  deficiencies  negate  the  validity  of 
comparisons  between  TLEs  developed  by  Fairchild  and  MLEs 
computed  by  the  Air  Force.  Additionally,  the  problems 
encountered  by  the  Air  Force  in  applying  the  model  cast 
serious  doubt  upon  the  ability  to  make  valid  TLE/MLE  com- 
parisons. This  is  illustrated  in  Figure  6. 
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THEN: 


TLE/MLE  COMPARISON  -+  INVALID 

Where : 

F = Fairchild 
AF  = Air  Force 
= Consistent 

DATA  = Data  used  for  OS.S  cost  projections 
LRU  = Line  Replaceable  Units 
METHOD  = Method  of  Application 


Figure  6 

Actual  Comparison  of  Contractor  TLEs 
and  Air  Force  MLEs 
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Chapter  7 


ACCOMPLISHMENT  OF  AIR  FORCE 
OBJECTIVES 


Introduction 

Chapters  4 — 6 of  this  thesis  analyzed  the  basic  tools 
required  to  apply  the  O&S  cost  model:  data,  TLE  computa- 
tions, and  MLE  computations.  The  conclusions  reached  in 
those  chapters  will  now  be  applied  to  determine  if  the  Air 
Force  objectives  established  in  Chapter  3 were  accomplished 
by  O&S  cost  model  application  to  the  A-X/A-10  acquisition. 
Additional  information,  where  needed,  will  be  presented  in 
order  to  make  this  determination.  Each  objective  will  be 
evaluated  separately  and  tested  against  the  criteria  estab- 
lished in  the  Methodology  and  restated  below: 

1.  The  model  must  have  been  employed  toward 
accomplishing  this  objective. 

2.  The  data  and  techniques  usee  in  applying  the 
O&S  cost  model  must  have  resulted  in  valid  or  accurate 
information  for  use  in  accomplishing  this  objective. 


The  Overall  Objective 

As  stated  in  Chapter  3,  the  overall  objective  of 
O&S  cost  model  application  was  to  instill  in  the  contractors 
the  incentive  to  consider  logistics  effects  (O&S  costs)  in 


their  designs  of  the  A-X  aircraft.  By  stating  their  intent 
to  use  this  model  in  the  RFP  (2;  3)  and  subsequently  carry- 
ing out  this  intent  through  actual  use  of  the  model  (1),  the 
Air  Force  fulfilled  the  requirement  established  in  criterion 
number  1. 

Testing  the  second  criterion  was  difficult  regarding 
this  objective  because  of  its  general  nature.  The  motiva- 
tion of  the  contractors  to  emphasize  O&S  cost  considerations 
was  not  dependent  upon  the  results  of  O&S  cost  model  appli- 
cation, but  rather  a product  of  the  knowledge  that  the  Air 
Force  was  considering  ownership  costs  a primary  factor  in 
A-X  acquisition  (13;  14;  29).  j 

Northrop  personnel  confirmed  that  their  company  did 
give  more  consideration  to  O&S  costs  because  of  the  appli- 
cation of  the  O&S  cost  model.  One  Northrop  executive  stated 
that  O&S  cost  model  application  had  a distinct  impact  on 
support  costs  of  the  airplane  (13) . Another  stated  that 
although  O&S  costs  were  always  considered  in  aircraft 
designs,  from  the  standpoint  of  compromising  weight  increases 
in  the  aircraft  for  increased  R&M,  the  O&S  cost  model  made 
a difference.  He  further  stated  that  the  cost  plus  incen- 
tive contract  combined  with  O&S  cost  model  application 
motivated  the  contractor  toward  better  design  (14).  A 
Fairchild  representative  expressed  similar  views  concerning 
the  influence  of  the  O&S  cost  model  on  aircraft  design.  He 
stated,  "Much  of  the  aircraft  design  was  governed  by 


design-to-cost  constraints  and  by  support  cost  considerations 
in  the  O&S  cost  model  [29]." 

Tf  greater  consideration  was  given  to  O&S  costs  in 
the  design  of  the  A-10  aircraft,  it  would  logically  follow 
that  the  aircraft  has  proven  itself  in  terms  of  reliability 
and  maintainability.  Since  the  beginning  of  its  operational 
flying,  the  A-10  has  exhibited  a high  rate  of  operational 
readiness  (OR).  In  fact,  according  to  the  A-10  SPO  director, 
the  "reliability  of  the  system  is  almost  twice  that  projected 
for  maturity  [4]." 

The  evidence  provided  above  is  sufficient  to  support 
the  fact  that  the  contractors  were  motivated  by  O&S  cost 
model  application  to  consider  support  costs  in  the  design 
of  the  A-X  aircraft.  Further,  this  consideration  has 
resulted  in  a weapon  system  that  has  demonstrated  a higher 
OR  rate  than  projected.  These  facts  lead  to  the  conclusion 
that  the  overall  objective  of  O&S  cost  model  application 
was  accomplished. 

Use  of  the  Model  in 
Source  Selection 

The  next  objective  to  be  evaluated  is  the  use  of 
the  O&S  cost  model  TLE  computations  as  a criterion  for 
source  selection.  The  members  who  sat  on  the  source  selec- 
tion board  could  not  be  contacted  for  interviews?  therefore, 
the  research  team  relied  upon  document  research  to  determine 
if  the  O&S  cost  model  was  employed  toward  the  selection  of 
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the  winning  contractor.  A study  conducted  by  the  Joint 
AFSC/AFLC  Commanders'  Working  Group  on  Life  Cycle  Costing 
states,  "During  the  proposal  evaluation  period,  the  data 
necessary  for  a thorough  Air  Force  validation  of  the  contrac- 
tors' bid  target  logistic  effects  were  not  available  [35:11]." 
Therefore,  any  comparison  made  by  the  Source  Selection  Board 
between  contractor  TLEs  would  have  been  of  questionable 
value.  Additionally,  it  has  already  been  established  in 
Chapter  5 that  TLE  computations  cannot  be  meaningfully  com- 
pared because  of  the  different  criteria  used  by  each  con- 
tractor in  selecting  LRUs  to  be  evaluated. 

Finally,  investigation  of  TLE  values  has  revealed 
that  the  TLE  total  submitted  by  Northrop  Corporation 
($821.3  million)  was  approximately  $19  million  less  than 
that  submitted  by  Fairchild  ($840.3  million)  (9;  21). 
Therefore,  it  is  clear  that  the  Source  Selection  Board  did 
not  consider  the  TLEs  computed  by  the  contractors  as  a sig- 
nificant basis  for  source  selection.  Hence,  the  Air  Force 
objective  to  use  the  application  of  the  O&S  cost  model  as  a 
criterion  for  source  selection  was  not  accomplished. 

Use  of  the  Model  in 
ECP  Evaluation 

The  intent  to  use  the  O&S  cost  model  to  evaluate 
ECPs  has  been  firmly  established  in  Chapter  3.  The  methods 
by  which  this  was  to  be  done  and  the  success  of  this  appli- 
cation of  the  model  will  be  addressed  now. 


The  detailed  procedures  for  preparing  and  presenting 
O&S  cost  data  for  ECP  changes  are  spelled  out  in  an  Engi- 
neering Release  (ER)  published  by  Fairchi Id-Republic 
Company  in  April, 1977.  The  procedures  are  summarized  in  a 
later  document  published  by  Fairchild: 

To  permit  this  evaluation  of  Logistics  effects,  the 
contractor's  analysis  of  each  Class  I ECP  shall  include 
a comparison  of  the  differences  in  TLE  values  as  com- 
puted for  the  baseline  weapon  system  configuration 
versus  that  resulting  under  the  proposed  ECP  .... 

Upon  notification  of  an  approved  ECP,  the  O&S  costing 
activity  posts  the  delta  values  for  each  of  the  thirteen 
model  elements  on  the  matrix  log  previously  prepared  to 
show  the  baseline  changes  [9:7-8]. 

That  the  model  was  actually  used  in  this  manner  is  confirmed 

by  the  following  statement: 

To  date,  approximately  65  approved  changes  have 
been  added  to  the  matrix,  with  a net  increase  of 
approximately  four  percent  to  the  baseline  total  cost 
prediction  [9:8]. 

Having  established  that  the  O&S  cost  model  was  used 
to  evaluate  the  cost  impacts  of  ECPs,  the  results  provided 
by  this  evaluation  will  now  be  examined.  The  SPO  was  con- 
tacted to  determine  how  successful  the  use  of  the  O&S  cost 
model  was  in  this  regard.  This  investigation  revealed  that 
the  SPO  had  initiated  action  in  March,  1978,  to  discontinue 
this  particular  application  of  the  model  in  favor  of  another 
model  provided  in  the  "Logistics  Support  Cost  Model  User's 
Handbook"  (33).  This  decision  was  the  result  of  an  in-depth 
study  by  SPO  personnel  of  the  effectiveness  of  using  the 
O&S  cost  model  in  ECP  evaluation  (25).  The  conclusion  of 


this  study  was  that: 


The  current  [O&S]  model  is  not  a good  tool  to  use 
for  ECP  purposes  because: 

a.  Some  of  the  equations  are  not  sufficiently 
explicit  (e.g..  Peculiar  Support  Equipment). 

b.  The  equations  do  not  accurately  reflect  pres- 
ent Air  Force  operations  (e.g.,  spare  safety  levels, 
equipment  usage) . 

c.  Air  Force  supplied  parameter  values  and 
constants  are  out  of  date  and  unrealistic  when  compared 
to  current  programmed  flying  hours,  aircraft  inventory 
and  airframe  life  projections. 

d.  Contractor  provided  parameter  values  and 
constants  are  also  unrealistic  [33:2]. 

The  originator  of  this  study  further  states: 

If  this  update  to  the  O&S  cost  impacts  on  ECPs  is 
not  accomplished  we  will  continue  to  receive  a mis- 
leading, inaccurate,  1970  baselined  assessment  in 
future  analyses  [33:2], 

Based  on  this  information,  the  only  logical  conclu- 
sion regarding  the  use  of  the  O&S  model  as  a tool  in  evalu- 
ating ECPs  is  that  the  model  is  ineffective.  Therefore, 
the  objective  of  using  the  O&S  cost  model  for  this  purpose 
was  not  accomplished. 

Use  of  the  Model  in 
Incentive  Award 
Determination 

To  determine  the  actual  role  of  the  O&S  cost  model 
in  making  an  award  fee  determination,  the  A-10  SPO  director, 
who  served  as  a member  of  the  Incentive  Award  Fee  Board,  was 
interviewed.  He  stated  that  the  degree  to  which  the  O&S 
cost  model  was  used  is  proprietary  information.  However, 

"a  very  large  part  of  the  incentive  award  fee  was  based  on 
TLE/MI.E  comparisons  [4]."  This  is  sufficient  to  conclude 
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that  criterion  number  1,  regarding  the  use  of  the  model 
toward  this  objective,  was  met. 

Chapter  6 is  an  in-depth  study  of  the  validity  of 
TLE/MLE  comparisons.  Due  to  extensive  problems  with  data 
availability  and  model  application  in  this  area,  it  has 
been  determined  that  a valid  comparison  between  TLEs  and 
MLEs  cannot  be  made. 

Another  factor  that  warrants  mention  here  is  that 
a large  part  of  the  contractor's  incentive  to  compute  TLEs 
as  accurately  as  possible  was  compromised  by  the  method  in 
which  the  model  was  employed  in  this  area.  Project  ABLE 
stipulates  that  TLEs  are  to  be  computed  during  the  competi- 
tive pha::e  of  the  acquisition  process  to  insure  that  they 
are  not  inflated  (4:2-6).  Research  has  revealed  that 
Fairchild  was  instructed  by  the  USAF  to  recompute  its  TLE 
figures  on  at  least  four  occasions  subsequent  to  contract 
award  (March  1973).  The  results  of  these  computations  are 
presented  in  Table  3. 


TABLE  3 

FAIRCHILD-REPUBLIC  TARGETED  LOGISTICS 
EFFECTS 


TLE 

DATE 

AMOUNT  ($  MILLION) 

Revised 

25  Jun  73 

819.5 

Baseline 

30  May  74 

836.2 

Baseline  + ECP 

1 Apr  76 

832.8 

Final 

30  Apr  77 

919.2 

Each  of  these  changes  was  a result  of  some  government  action 
regarding  changes  to  the  rules  by  which  the  MLEs  would  be 
computed  (9:7,9).  The  fact  that  these  revisions  to  TLEs 
were  made  subsequent  to  contract  award  negates  the  competi- 
tive incentive  to  keep  these  figures  as  low  as  possible.  In 
fact,  it  is  now  to  the  contractor's  advantage  to  make  these 
estimates  as  high  as  possible  in  order  to  enhance  the  chances 
of  a favorable  comparison  to  MLEs  computed  by  the  government 
(4:2-6) . 

Although  the  O&S  model  was  a major  consideration  in 
the  award  fee  determination,  the  inability  to  make  valid 
TLE/MLE  comparisons  because  of  the  factors  presented  above 
and  in  Chapter  6 casts  very  serious  doubt  on  the  usefulness 
of  the  O&S  cost  model  in  this  area.  Therefore,  it  is  con- 
cluded that  the  objective  of  using  the  O&S  cost  model  to  aid 
in  the  determination  of  an  award  fee  was  not  accomplished. 


Conclusions 

The  application  of  the  O&S  cost  model  to  the  acqui- 
sition of  the  A-10  did  result  in  a better,  more  reliable 
weapon  system  through  the  accomplishment  of  the  overall 
objective  as  stated  earlier. 

Due  to  problems  encountered  with  obtaining  data 
and  model  application  deficiencies,  it  has  been  determined 
that  none  of  the  other  Air  Force  objectives  in  O&S  cost 
model  application  were  accomplished.  Unless  steps  are  taken 
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to  correct  these  problems,  the  accomplishment  of  the  overall 

objective  by  future  users  will  be  jeopardized.  This  was 

confirmed  by  one  Fairchild  representative  who  stated  that 

any  future  emphasis  on  the  use  of  this  model  by  his  company 

will  require  increased  efforts  by  the  Air  Force  to  correct 

»•  • 

deficient  areas  (29).  The  last  chapter  of  this  thesis  will 
present  recommendations  for  correcting  these  deficiencies. 
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Chapter  8 

CONCLUSION  AND  RECOMMENDATIONS 

The  O&S  cost  model  can  be  a valuable  tool  for 
increasing  the  reliability  and  maintainability  of  new 
weapon  systems.  It  has  proven  its  worth  in  motivating  con- 
tractors to  consider  cost  of  ownership  in  the  design  of 
new  aircraft.  It  is  clear  that  the  first  application  of 
the  model  had  problems.  For  improved  future  use,  these 
problems  must  be  analyzed  and  resolved.  Analyses  of  the 
problem  areas  appear  in  Chapters  4 through  7.  This  chapter 
is  dedicated  to  recommendations  for  resolving  problems  in 
the  areas  of  data  availability,  TLE/TLE  comparisons,  TLE/MLE 
comparisons,  and  accomplishment  of  Air  Force  objectives. 
Additionally,  conclusions  are  drawn  concerning  the  future 
use  of  this  O&S  cost  model.  It  is  not  intended  that  these 
recommendations  be  considered  collectively  exhaustive  or 
mutually  exclusive;  the  need  for  further  research  is 
examined  in  the  final  pages  of  this  chapter. 

Data  Availability 

As  stated  earlier  (in  Chapter  4),  the  A-X/A-10  O&S 
cost  model  cannot  be  exercised  strictly  on  the  basis  of 
data  obtained  from  the  D056  Product  Performance  Collection 
System.  Problems  encountered  by  the  Air  Force  and 
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contractors  preventing  achievement  of  this  objective  are 
summarized  below. 

1.  Off-equipment  TLE  calculations  were  hampered 
by  the  absence  of  depot-level  repair  data  in  D056. 

2.  Work  Unit  Codes  were  not  line  item  identifiers; 
more  than  one  FSN  was  reported  against  a single  WUC. 

3.  There  was  no  acceptable  cross-reference  system 
to  convert  contractor  Aeronautical  Equipment  Reference 
Number  (AERNO)  or  Federal  Stock  Number  (FSN)  to  Work  Unit 
Code  (WUC) . 

A suggested  means  of  solving  the  first  problem  con- 
cerning the  absence  of  depot-level  data  is  provided  in  the 
"TLE/MLE  Comparisons"  section  of  this  chapter.  Resolution 
of  problems  two  and  three  require  action  on  the  part  of  the 
Integrated  Logistics  Support  (ILS)  Branch  of  the  concerned 
SPO.  The  ILS  Branch  must  insure  that  each  WUC  has  its  own 
unique  stock  number.  This  will  probably  increase  the 
number  of  work  unit  codes,  but  it  will  provide  the  identi- 
fication of  each  unique  LRU.  LRUs  that  consist  of  stock 
numbered  subassemblies  are  the  largest  contributors  to  the 
problems  encountered  in  the  "line-item-identifier"  concept 
in  the  A-X/A-10  O&S  cost  model.  An  error  listing  that 
compares  a previous  month's  stock  number  and  WUC  to  the 
current  month's  stock  number  and  WUC  should  be  used  to  flag 
ch.inoes  in  these  iei.il  ionships.  This  would  he  more  eosl 
elloctive  than  screening  the  entire  document.  01  course, 


this  technique  would  be  effective  only  after  the  initial  run 
is  evaluated  and  corrected.  Additionally,  a list  of  cross- 
referenced  AERNOs  and  FSNs  should  be  solicited  from  the 
contractor  and  used  to  further  cross-reference  with  Air 
Force  WUCs.  Another  related  problem  was  the  lack  of  defi- 
nition among  line  item  identifiers  and  prices  (see  Chapter  7) . 
The  aforementioned  technique  would  insure  that  each  price 
would  be  specific  for  each  line  item.  A clear  distinction 
would  then  be  made  between  assemblies  and  subassemblies. 

TLE/TLE  Comparisons 

Chapter  5 cited  problems  concerning  the  selection  of 
LRUs  which  led  to  the  inability  to  make  a valid  comparison 
between  contractor  TLEs . The  specific  problems  cited  were: 

1.  The  contractors  failed  to  use  the  same  criteria 
for  LRU  selection. 

2.  The  Air  Force  failed  to  specifically  define  the 
terms  "LRU"  and  "high  cost". 

The  selection  of  LRUs  is  the  most  important  step 
in  TLE  computations.  The  seemingly  infinite  list  of  LRUs 
that  comprise  an  entire  weapon  system  must  be  scrutinized 
by  each  contractor  to  determine  which  ones  are  significant 
enough  to  be  considered  high  cost  drivers.  To  add  to  this 
problem,  the  contractor  does  not  want  to  include  any  LRUs 
that  might  be  unique  to  his  aircraft  thereby  driving  his 
TLE  figure  higher  than  his  competitor.  The  Auxiliary  Power 
Unit  (APU)  on  the  A-9  is  a good  example  of  this.  Northrop 


included  an  APU  in  its  original  design  whereas  Fairchild 
did  not  (14).  O&S  costs  for  the  APU  were  included  in 
Northrop' s TLE  computations  and  no  comparable  LRU  was  con- 
sidered in  Fairchild's  TLE.  The  contractors  were  asked  how 
LRU  selection  might  be  standardized.  Northrop  recommended 
that  each  contractor  submit  a list  of  LRUs  to  the  Air  Force 
and  that  the  Air  Force  then  discriminate  between  similar 
LRUs  and  peculiar  LRUs.  The  similar  LRUs  could  then  be 
compared  directly  in  terms  of  TLEs  whereas  the  peculiar  LRUs 
could  be  evaluated  separately  in  terms  of  O&S  costs  and 
improved  performance  (14).  Fairchild  suggested  that  only 
the  top  fifty  or  100  O&S  cost  drivers  of  each  aircraft  be 
evaluated  via  the  O&S  cost  model  (39).  Both  of  these 
suggestions  are  feasible  and  both  would  standardize  the 
LRU  selection  process,  although  Fairchild’s  suggestion  may 
not  provide  the  degree  of  O&S  cost  coverage  that  other 
alternatives  would. 

Research  has  revealed  another  possible  solution  to 
the  problem  of  LRU  selection  which  is  recommended  by  this 
research  team.  Northrop 's  analysis  of  historical  data  of 
inventory  aircraft  revealed  that  15  percent  of  all  LRUs 
account  for  80  percent  of  the  total  O&S  costs  (21:8).  The 
Air  Force  should  compile  a list  of  these  LRUs  and  submit  a 
copy  to  each  contractor.  The  contractor  should  then  develop 
a list  of  LRUs  for  his  aircraft  which  are  comparable  to 
those  on  the  Air  Force  list.  From  the  contractors'  lists, 
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the  Air  Force  could  then  develop  a final  list  of  LRUs  to 
be  evaluated  regarding  O&S  costs.  In  this  way  the  Air  Force 
could  insure  compatibility  of  TLEs  for  comparison  without 
considering  the  total  O&S  costs  for  the  entire  weapon 
system.  This  would  also  solve  the  problem  of  defining  the 
terms  "LRU"  and  "high  cost,"  for  discrimination  in  these 
areas  would  now  be  in  the  hands  of  the  Air  Force. 

If  the  preceding  solution  is  not  employed  however, 
the  problem  of  defining  terms  "LRU"  and  "high  cost"  will 
still  exist.  The  definition  of  LRU  provided  to  the  con- 
tractors states: 

A component  of  a system  that  is  designed  for 
removal  when  it  malfunctions  and  is  replaced  with  a 
like  unit  by  operating  squadron  personnel  performing 
on-equipment  maintenance  on  board  the  aircraft  or  on 
the  flight  line  [30:2]. 

Fairchild  pointed  out  that  this  could  apply  to  just  about 
any  system  on  the  aircraft  (see  Chapter  5,  page  39).  Fair- 
child  solved  this  problem  by  adding  the  word  "normally"  to 
the  definition  (30:5).  If  both  contractors  had  done  this, 
the  problem  may  have  been  solved.  A better  solution,  how- 
ever, would  be  for  the  Air  Force  to  scrutinize  the  contrac- 
tors' proposed  LRUs  and  make  the  determination  as  to  whether 
or  not  the  items  are,  in  fact,  LRUs.  The  solution  to  the 
problem  of  defining  "high  cost"  can  be  resolved  by  simply 
stipulating  a dollar  value  in  terms  of  cost  to  maintain  or 
replace  the  item  over  the  projected  life  cycle  of  the  weapon 
system. 
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If  these  recommendations  had  been  instituted  in  the 
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initial  application  of  the  O&S  cost  model,  the  comparison 
between  contractor  TLEs  would  have  been  meaningful.  Future 
implementation  of  these  recommendations  will  enhance  the 
probability  that  a valid  comparison  between  TLEs  can  be 
made . 

TLE/MLE  Comparisons 

Comparisons  between  Fairchild's  TLEs  and  Air  Force's 
MLEs  were  determined  to  be  invalid  in  Chapter  6 due  to  the 
following  problems: 

1.  Lack  of  sufficient  data  on  the  part  of  the 
contractor . 

2.  Lack  of  clear  definition  concerning  what  costs 
to  include  and  what  costs  to  exclude. 

3.  Lack  of  sufficient  unit  price  data  on  the  part 
of  the  Air  Force. 

4.  Inability  to  track  and  aggregate  test  removals. 

5.  The  need  to  make  manual  adjustments  to  the 
computer  product  for  MLEs  1-4. 

The  lack  of  sufficient  data  for  contract  computation 
of  TLEs  lies  mainly  in  the  areas  of  on-equipment  maintenance 
and  depot  repair  man-hours.  In  instances  where  data  is 
unavailable,  it  is  recommended  that  the  Air  Force  establish 
numerical  constants  to  be  used  in  the  O&S  cost  model 
equations.  For  example,  such  constants  may  be  supplied  in 
the  area  of  depot  repair  man-hours.  While  this  solution 
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would  not  insure  accuracy  of  TLE/MLE  computations,  as  long 
as  the  same  numbers  were  used  by  the  Air  Force  and  the  con- 
tractor, a valid  relative  comparison  could  be  made.  The 
number  of  items  going  to  depot  for  repair  would  become  the 
major  variable  in  these  calculations  and  would  be  the  major 
reason  for  TLE  and  MLE  differences.  A less  specific,  but 
related,  recommendation  for  solving  this  problem  is  pre- 
sented by  the  contractor  in  a Fairchild  publication: 

The  recommendation  here  is  that  if  depot  repair 
costs  must  be  included,  more  recognition  should  be 
given  to  data  non-availability  and  prior  determination 
specified  relative  to  measurement  methods  and  status 
[9:121 • 

Improved  communication  between  the  contractors  and  the  USAF 
is  important  in  solving  the  problem  of  providing  sufficient 
data  to  the  contractor.  A management  structure  which  will 
facilitate  the  implementation  of  the  recommendations  stated 
here  is  presented  in  a later  portion  of  this  chapter.  Such 
a structure  will  also  provide  for  more  precise  definition 
of  costs  to  include  and  costs  to  exclude  when  computing  TLEs . 

The  problem  of  unit  price  availability  was  largely 
a result  of  the  short  period  allowed  for  validation  (5,287 
hours).  A partial  solution  to  this  problem  would,  therefore, 
be  to  extend  the  validation  period.  It  is  recommended  that 
future  validation  periods  be  set  at  no  less  than  10,000  hours 
of  operational  flying  time.  This  will  allow  more  time  for 
pricing  information  to  be  gathered  (7:6).  A disadvantage 
of  extending  the  validation  period  is  that  the  contractor 
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would  have  to  wait  a longer  period  for  the  award  fee. 

General  Brill  recommended  that,  to  solve  this  problem,  the 
award  fee  might  be  granted  in  phases.  For  example,  a 
certain  percentage  could  be  granted  at  the  end  of  5000 
hours,  a larger  percentage  at  7000  hours  and  so  forth.  The 
percentages  of  possible  award  fet  could  be  increased  as 
information  became  more  reliable  (4). 

Another  aspect  of  the  uni  price  problem  is  that  of 
updating  prices.  Here  again,  the  management  structure 
proposed  in  this  chapter  will  provide  communication  channels 
to  insure  that  price  information  required  for  the  computa- 
tion of  MLEs  1-4  is  kept  current.  The  inability  to  aggre- 
gate test  removals  beyond  the  quarterly  period  is  a problem 
which  is  also  addressed  in  Chapter  6.  It  is  recommended 
that,  since  data  is  not  maintained  for  a long  enough  period 
to  properly  aggregate  this  information,  all  calculations  be 
done  on  a quarterly  basis.  The  summation  of  quarterly  data 
for  each  MLE  (MLEs  1-4)  could  then  be  accomplished  for  as 
many  quarters  as  the  validation  period  may  require.  This 
would  necessitate  minor  changes  in  the  current  computer 
program  which  performs  these  calculations. 

The  final  problem  revealed  by  this  investigation  in 
the  area  of  TLE/MLE  comparisons  is  that  of  having  to  make 
manual  changes  in  the  computer  program.  Many  of  these 
changes  were  necessitated  by  problems  with  unit  prices  and 
WUCs  not  identifying  the  proper  subassemblies  (see  Chapter  6) 
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Closer  monitoring  of  maintenance  reports  is  required  in  this 
area  to  preclude  such  problems.  Closer  monitoring  will  be 
possible  with  the  implementation  of  the  new  management 
structure . 

The  final  problem  considered  in  the  area  of  TLE/MLE 
comparisons  is  that  of  adjusting  TLE  values  after  contract 
award.  As  stated  previously,  this  negates  the  contractor's 
incentive  to  keep  his  TLE  values  as  low  as  possible.  It  is 
recommended  that  the  TLE/MLE  comparison  should  be  made 
based  upon  original  LRUs  only.  Any  ECPs  would  not  be  con- 
sidered in  this  comparison.  Any  LRUs  that  have  been  deleted 
due  to  ECPs  could  simply  be  removed  from  the  contractor's 
original  TLE  and  not  considered  by  the  Air  Force  in  its 
MLE  computation.  There  are  two  advantages  to  be  gained 
here : 

1.  The  award  fee  will  be  based  on  TLE  values  that 
were  computed  prior  to  contract  award. 

2.  The  contractor  will  not  be  penalized  for 
recommending  ECPs  to  improve  aircraft  performance. 

Implementation  of  the  recommendations  presented  in 
this  section  will  enhance  the  ability  of  an  Evaluation 
Board  to  make  valid  comparisons  between  TLEs  and  MLEs . The 
ability  to  implement  these  recommendations  is  largely 
dependent  upon  the  development  of  a project  management 
structure  that  will  provide  the  means  for  such  implementation. 
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Project  Management 
of  O&S  Cost  Model 
Application 

The  importance,  complexity,  and  effort  of  O&S  cost 
model  application  leads  to  the  conclusion  that  it  is  , in 
and  of,  itself  a project  which  should  be  separately  managed 
within  the  SPO.  To  obtain  accuracy  in  the  calculations  of 
TLEs  and  MLEs  requires  constant  management  and  monitoring 
at  all  levels  involved  with  the  maintenance  of  the  weapon 
system.  An  organizational  structure  for  project  management 
of  O&S  costing  will  be  presented  in  this  chapter.  The 
advantages  of  project  management  cannot  be  overstated.  The 
structure  proposed  here  will  increase  the  SPO's  ability  to 
monitor  this  aspect  of  the  acquisition  process.  It  will  do 
this  with  a minimum  increase  in  manpower  and  without  requir- 
ing changes  in  the  current  data  system.  Further,  the 
development  of  such  a management  structure  will  demonstrate 
to  the  contractors  that  the  Air  Force  is  seeking  to  improve 
its  ability  to  administer  life  cycle  costing  techniques  in 
the  acquisition  process,  thereby  increasing  the  contractors' 
motivation  in  this  area.  Channels  of  communication  for 
handling  various  problem  areas  will  be  illustrated.  This 
research  team  firmly  believes  that  improved  management 
techniques  are  the  key  to  successful  administration  of  the 
O&S  cost  model. 

The  main  personnel  involved  with  the  application  of 
the  O&S  cost  model  are: 
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1.  Contractor  O&S  model  personnel. 


2.  Integrated  Logistics  Support  (II. S)  Branch  of 

the  SPO. 

3.  Item  Managers  (IM)  at  ALCs. 

4.  Production  Analysis  Section,  Deputy  Commander 
for  Maintenance  (DCM) . 

5.  ALC  personnel  involved  with  the  computer  aug- 
mented portion  of  the  model. 

It  is  recommended  that  a project  management  structure  be 
established  in  accordance  with  Figure  7. 


Figure  7 

Project  Management  Structure 


The  SPO  O&S  Cost  Manager  will  be  the  center  of 
communication  among  all  agencies  and  the  only  communication 
point  with  the  contractors.  The  Item  Managers  will  be 
required  to  report  all  price  and  LRU  changes  to  the  SPO  O&S 


changes.  This  channel  of  communication  would  insure  that 
price  and  WUC  information  is  kept  current  in  the  computer 
augmented  portion  of  the  O&S  cost  model. 
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Constant  communication  is  required  between  the 
Production  Analysis  Section,  Deputy  Commander  for  Mainte- 
nance, and  the  SPO  O&S  Cost  Manager  to  insure  that  WUC  and 
man-hour  information  is  being  accurately  reported. 

Screening  of  the  maintenance  reports  received  from  the 
flight  line  should  be  mandatory  in  the  case  of  new  systems 
being  evaluated  by  the  O&S  cost  model.  Presently,  such 
screening  is  optional  (36:3-1).  If  problems  are  detected 
by  the  SPO  Manager,  immediate  action  should  be  taken  to 
correct  them.  An  example  of  such  a problem  existed  in  the 
1300  WUC  category  of  the  A-10.  The  1300  WUC  encompasses 
wheels,  brakes,  and  tires.  In  the  A-10  application  of  the 
model,  tire  changes  were  being  misreported  as  removals  of 
the  entire  assembly.  This  channel  of  communication  will 
help  to  alleviate  problems  concerning  WUC  and  -FSN  identifi- 
cation mentioned  in  the  "Data  Availability  From  D056" 
portion  of  this  chapter. 

With  this  management  structure,  the  SPO  O&S  Manager 
can  maintain  constant  communication  with  the  prime  ALC 
computer  program  monitors  to  insure  that  all  problems  with 
computer  inputs  and  programming  are  resolved  immediately. 

Finally,  the  contractor  will  have  a definite  source 
to  go  to  to  resolve  problems  with  obtaining  data,  i.o.,  the 
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O&S  Cost  Manager  in  the  SPO.  The  Cost  Manager  can  rely  on 
his  expertise  or  defer  to  one  of  the  other  sources  available 
to  him  to  obtain  information  for  the  contractors. 


All  personnel  involved  in  this  program  management 
structure  must  be  thoroughly  briefed  on  the  essential  part  < 

that  they  will  play  in  O&S  cost  model  application.  Adoption 
of  this  project  management  plan  will  facilitate  the  imple- 
mentation of  the  other  recommendations  presented  in  this 
chapter. 

Future  Use  of  O&S 
Cost  Model 

It  is  necessary  to  assess  whether  or  not  the 
original  Air  Force  objectives  can  be  accomplished  through 
implementation  of  these  recommendations. 

Overall  objective.  It  has  been  established  that  the  overall 
objective  of  model  application  was  accomplished  in  its 
initial  application.  It  has  also  been  determined  that  the 
credibility  of  model  validity  has  declined  in  the  eyes  of 
the  contractors  due  to  its  deficiencies.  It  is  logical  to 
conclude  that  correcting  these  deficiencies  is  essential  to 
the  accomplishment  of  this  objective  in  the  future. 

Objective  in  source  selection.  The  objective  of  using  the 
O&S  model  as  a source  selection  criterion  is  a realistic  one 
which  can  be  accomplished  by  implementing  the  recommendations 
presented  in  this  chapter.  Standardization  of  LRU  selection 
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and  improved  data  availability  will  insure  accomplishment 
of  this  objective. 

Objective  in  ECP  evaluation.  Use  of  the  O&S  cost  model  to 
evaluate  ECPs  has  proven  to  be  an  unrealistic  objective. 

The  SPO's  research  presented  in  Chapter  7 affirms  this 
conclusion.  Therefore,  it  is  recommended  that  the  O&S  cost 
model  not  be  used  toward  this  end.  Other  models  are  more 
appropriately  designed  for  ECP  cost  impact  studies. 

Objective  in  award  fee  determination.  The  O&S  cost  model 
was  not  an  effective  tool  in  making  an  award  fee  determina- 
tion in  the  A-10  case.  It  can,  however,  be  used  effectively 
toward  this  end  if  the  recommendations  submitted  in  this 
chapter  are  instituted.  Better  data  management,  improved 
contractor-SPO  communication  and  procedural  changes  in 
Air  Force  computation  techniques  will  lead  to  valid  TLE/MLE 
comparisons.  This,  in  turn,  will  result  in  the  successful 
accomplishment  of  this  objective. 

Recommendations  for 
Further  Study 

During  the  course  of  this  research,  several  areas 
have  been  touched  upon  which  were  outside  the  scope  of 
this  study,  but  whicn  warrant  closer  examination.  Support 
should  be  provided  for  further  research  in  the  following 
areas : 
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1 . A methodology  for  gathering  the  data  required 
for  Ot.r;  cost  model  application  should  be  developed.  Instead 
of  providing  contractors  with  access  to  the  66-1  MDC  system, 
specific  portions  of  this  system  should  be  interrogated  to 
extract  only  the  information  needed  to  exercise  the  model. 
Procedures  for  facilitating  this  process  would  save  time 
and  insure  complete  data  availability. 

2.  The  program  for  computing  MLEs  1-4  should  be 
evaluated  and  modified  as  necessary  to  preclude  manual 
changes  and  computational  deficiencies.  Modifications 
must  be  made  to  insure  proper  aggregation  of  test  removals 
and  support  general  data  incorporation  into  the  program. 

3.  A comparative  analysis  between  application  of 
the  O&S  cost  model  to  the  F-16  acquisition  and  the  A-10 
acquisition  should  be  conducted. 
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4. 


INTERVIEW  GUIDE  FOR  SPO  (AFSC),  AFALD , 

AND  HQ  USAF/BRMC 

1.  What  were  the  objectives  of  the  O&S  cost  model  as 
applied  to  the  A-10? 

2.  Did  your  office  participate  in  its  development? 

3.  What  are  your  recommendations  for  future  application 
of  the  O&S  cost  model? 

4.  How  does  the  AF  define  LRU? 

5.  How  was  the  model  used  to  fulfill  the  AF  objectives? 

6.  What  data  was  supplied  to  the  contractors  for  the  com- 
putations of  TLEs? 

7.  What  data  was  used  to  compute  MLEs  and  what  techniques 
were  used  for  these  computations? 

8.  Do  you  have  any  recommendations  for  changes,  additions 
or  deletions  in  the  model  before  it  is  applied  in  future 
applications? 

9.  How  effective  is/was  the  model  in  evaluating  ECPs? 

10.  What  techniques  were  used  to  apply  the  data  to  the 
mode 1 ? 
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INTERVIEW  GUIDE  FOR  CONTRACTORS 

1.  what  benefits,  or  disadvantages,  are  there  in  LCC 
application? 

2.  Did  the  O&S  cost  model  affect  your  design  of  the 
aircraft? 

3.  What  were  the  AF  objectives  in  applying  the  O&S  cost 
model  to  the  A-X/A-10  program? 

4.  What  was  your  role  in  designing  and  applying  the  O&S 

cost  model? 

5.  Did  the  model  become  an  end  in  itself? 

6.  Is  the  model  too  complex? 

7.  Where  did  the  data  come  from  to  develop  TLEs?  Do  you 
feel  the  data  was  valid?  Why?  Why  not? 

8.  How  did  you  define  LRUs?  Did  you  get  any  guidance 
from  the  A- 10  SPO? 

9.  What  are  your  recommendations  for  changes,  additions 
or  deletions  to  the  model  before  it  is  applied  to  future 
major  system  acquisitions? 

10.  How  effective  is/was  the  model  in  evaluating  ECPs? 

11.  What  techniques  were  used  to  apply  cost  data  to  the 
model? 
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UNCLASSIFIED 


INTERVIEW  GUIDE  FOR  ALC/AFLC 


1.  What  was  your  involvement  with  the  O&S  cost  model  as 
applied  to  the  A-X/A-10? 

2.  How  did  your  office  interface  with  the  contractors  and 
the  SPO  in  MLE  computation? 

3.  What,  if  any,  areas  of  the  model  are  managed  by  your 
office? 

4.  Is  the  model  viable? 

5.  What  does  your  office  use  the  model  for? 

6.  Did  you  participate  in  the  development  of  the  model? 

7.  Does  your  office  have  any  responsibility  for  the  model* 
application  to  the  A-10? 

8.  How  does  the  model  account  for  O&S  costs? 

9.  What  percentage  of  your  work  week  is  spent  on  O&S 
cost  related  items? 

10.  What  are  your  recommendations  for  changes  in  the 
model  before  it  is  applied  to  future  major  system  acqui- 


sitions? 
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MAINTENANCE  DATA  COLLECTION  (MDC) 

SYSTEM 

The  United  States  Air  Force's  Maintenance  Data 
Collection  System  is  utilized  at  all  levels  in  all  commands 
of  the  Air  Force.  Additionally,  the  data  contained  in  the 
MDC  is  provided  to  contractors  to  help  formulate  cost  esti- 
mates for  new  weapon  systems.  The  requirements  for  manage- 
ment support  of  MDC  are  provided  in  AFM  66-1,  Volume  II, 
Chapter  4,  Maintenance  Management . The  use  of  the  MDC 
system  includes: 

1.  Refining  inspection  programs 

2.  Determination  of  routine  modifications 

3.  Base  level  manning 

4.  Reliability  and  maintainability  studies 

5.  Configuration  management. 

AFM  66-267,  Maintenance  Data  Collection  System,  states  that 
maintainability  and  reliability  studies  demand  absolute 
data  accuracy. 

All  data  inputs  to  the  various  files  in  the  MDC 
system  are  input  from  AF  Form  1530,  Punch  Card  Transcript; 
AFTO  Form  349,  Maintenance  Data  Collection  Record;  AFTO 
Form  349-3,  Maintenance  Data  Collection  Record;  or  a tape 
file  from  command-unique  systems.  The  system  is  updated  by 
inputs  from  AF  Forms  1530  and  AFTO  Forms  349.  Both  these 
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forms  are  handscribed  documents  initiated  by  base  level 
maintenance  squadron  personnel.  Data  from  these  documents 
are  key  punched  daily  and  input  into  the  computer  files. 

Two  tapes  are  generated  each  month  for  MDC  data : a month- 
to-date  tape  (ABD3BA)  and  a history  tape  (ABD6DA) . 

The  monthly  MDC  transaction  listings  include: 

1.  On-equipment — PCN  SG001B501 

2.  Off-equipment — PCN  SG001B502 

3.  Support  General — PCN  SG001B503 

4.  Indirect  Labor — PCN  SG001B504 

5.  Parts  Replaced— PCN  SG001B505 

6.  Serially-controlled  and  Time  Change  Remove 
and  Replace — PCN  SG001B506 

7.  Engine  Remove  and  Replace — PCN  SG001B507. 

These  listings  are  the  basic  source  of  all  maintenance  tasks 
performed  and  direct  and  indirect  labor  expended  during  the 
month.  Analysis  by  work  unit  code  (WUC)  is  the  most  common. 
This  analysis  includes: 

1.  Isolation  of  frequent  malfunctioning  components — 
on  and  off  equipment  maintenance. 

2.  Forecasting  future  parts  requirements — parts 
replaced. 

3.  Auditing  time  change  requirements — serially- 
controlled  and  time  change  remove  and  replace. 

Additionally,  MDC  data  provides  inputs  for  a performance 
monitoring  system.  This  system  includes: 
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1.  WUC  Trend  Analysis  Report— PCN  SG001B732. 

This  report  includes: 

a.  total  number  of  failures 

b.  off-equipment  man-hours  (monthly) 

c.  on-equipment  man-hours  (as  a deviation  from 
12  month  historical  averages) . 

2.  Part  Number  Trend  Analysis  Report — PCN  SG001B734. 
This  report  includes  off-equipment  maintenance  information 
only. 

3.  On-Equipment  Performance  Hi-Lights — PCN 
SG001B771.  This  report  includes  each  WUC  that  has  received 
maintenance  that  exceeds  the  average  man-hours  per  unit. 
Average  man-hours  are  computed  on  a continuous  twelve  month 
cycle . 

4 . Off-Equipment  Performance  Hi-Lights — PCN 
SG001B772.  This  report  is  similar  to  on-equipment  except 
that  it  is  by  FSC. 

5.  High-25  On-Equipment  Failures — PCN  SG001B781. 
These  failures  are  identified  by  WUC.  Data  is  available  on 
a continuous  twelve  month  cycle. 

6.  High-25  Man-Hour  Consuming  Jobs — PCN  SG001B791. 
This  listing  includes  total  man-hours  for  on  and  off  equip- 
ment maintenance  by  WUC. 

To  utilize  this  data  for  any  given  system,  all  the 
data  tapes  must  be  interrogated  for  that  particular  system 
(A-10 , for  instance).  Both  monthly  and  historical  tapes 
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must  be  included  for  a meaningful  information  flow  to  be 
established.  The  users  may  then  utilize  this  data  for 
whatever  documentation/verification  necessary,  such  as 
computing  Measured  Logistics  Effects  (MLEs) , or  evaluating 
Engineering  Change  Proposals  (ECPs) . 
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